The incident report appeared to be encouraging. The request to pay resulted in a response of 201 Created. The webhook arrived at the target. The CRM received the message. The background process completed successfully without any errors. Until then, support accessed the customer’s account. The card was charged. However, the subscription was still listed as “trial.” The premium options were unavailable. The confirmation email was never sent. Finance had access to the invoice. However, the support was unable to locate the corresponding subscription. There was nothing broken. That was the trouble. All technical indicators were green, so it took the team one hour to search for something that was not there. A list of successful API calls will not confirm an integration for me. It shows that there was data flow between systems. But it does not confirm whether the customer got what he was promised by the company. This is the very reason why a lot of SaaS integrations fail. The endpoints were working properly, the payloads were correct, but the workflow gave a wrong result.
The Green Dashboard Is Answering the Wrong Question
The typical API monitor will be able to answer questions like: Did the call go through the server? How much time was required for the response? Is the endpoint returning a 2xx status code? These tests are good but insufficient for a multistep workflow involving multiple products. Take an example of upgrading to a new plan: the billing system will charge the user; the account system will upgrade the subscription; the product itself will unlock new features; the CRM will update the stage of the client; and the email system will send a confirmation. Six steps could be executed with one click. In the case where the first five have been successful and the last one has failed, there might be nothing wrong with the APIs yet the customer will perceive this as an unfinished upgrade. What is even worse is that all systems could potentially be carrying different truths at this point. Billing says “paid”. Product says “trial”. CRM says “customer”. Support needs to choose which of those records to consider while the customer is waiting. The important health indicator is not technical but rather business one: was the right plan created after the successful payment has been done?
Follow One Customer Journey From Start to Finish
Integration conversations tend to start with a diagram, where boxes are services, arrows are API calls, and everything looks neat. Customer journeys in reality behave differently. A single subscription includes a user, workspace, billing account, contract, payment method, and a couple of entitlements. Not all of those records have IDs, let alone agree on what an active customer is. The most straightforward way to spot integration risks is to trace the journey of one particular customer through all systems involved. Choose a new customer, upgrade, cancellation, or refund and make a list of records that are being created at every step along the way, their owner, and proof of completion of that step. Typically, assumptions that were previously invisible in the API documentation will be revealed in this exercise. One service identifies the customer by the email. Another identifies it via company ID. The billing service uses the subscription ID. The product needs tenant and user information. This system will serve its purpose until email addresses are changed, companies merge, or the same person becomes part of multiple teams. Then “send the customer to the CRM” becomes ambiguous. What customer? What company? What workspace? Which record is preserved during a merge operation? These types of questions are related to data modeling because they are indeed data modeling. The connector cannot resolve an identity problem the business has not yet solved.
Valid Data Can Still Mean the Wrong Thing
Schema validation works well for validating shape. One can use schema validation to validate that status is a string, amount is numeric, and renewal_date is in a valid format. One cannot use schema validation to validate whether the receiving system interprets these values in the same way. For instance, the term “Active” could mean different things in the billing application, the CRM system, and the product itself. The value of the amount field might be represented in dollars in one system and cents in another system. All three payloads can pass validation. It is from here that a clean integration begins to generate unexpected support inquiries. Although the data has arrived, its purpose has changed during the process. The way to fix this issue is to speak in simple terms. For every field that is of interest, provide a description of what the field means, which system owns it, and whether there is a possibility that other systems will overwrite it. Units of measurement, time zones, allowed transitions and interpretations of missing values must all be specified. I prefer to carry out this analysis before mapping code is written. It is easier to resolve a dispute as to what “canceled” means than to fix thousands of subscriptions later, if two systems understand it differently.
Time Is Part of the Data, Even When It Is Missing
APIs give structure to order of operations in development. First comes request A, then request B, then request C. In production, you add queues and retries, response time delays, worker restarts, and peaks in traffic. What arrives first doesn’t necessarily happen first anymore. Let’s say a customer upgrades at 10:02 and cancels at 10:04. The cancellation event arrives at the product first. The upgrade event is delayed, so it arrives five minutes later. The consumer that processes messages in the order of their arrival reactsivate an account that should remain canceled. No endpoint broke. It was just that the older event arrived at the last possible moment. Using timestamps can be useful, but they might not always be sufficient. Different services could use different clocks. While one timestamp may show the moment when the user clicked, another one could record the point in time when the job made the change. Some service providers overwrite their records, obscuring part of their history. Essential processes require clear ordering rules. Such rules can include a sequence number, an object version, an immutable event identifier, or a list of possible state changes. The consumer needs to understand that “cancelled to active” means that a new purchase needs to be made, not that any delayed update will do as long as it is valid JSON.
The Retry That Makes the Incident Worse
Retries are implemented to provide additional reliability. The request times out, so the job re-tries it. This is perfectly okay for a safe read. It is very risky for an operation which is associated with transactions like money transfer, sending a message, reserving inventory, or modifying access permissions. Let us take the refund example. The payment gateway processes it successfully but fails to deliver its confirmation before the support system can record it. The agent sees the timeout and performs the retry. If the payment gateway treats this as a new transaction, the customer gets two refunds. From the agent’s perspective, the retry was successful. From the financial perspective, the problem just became twice as big. An idempotency key makes each business activity unique and identifiable. Multiple attempts with the same key might generate the initial result rather than perform the action again. However, a reliable storage location for such keys is required. Storing the key in the memory of one person will be useless when the computer starts afresh and the other person continues with the task. In case of major operations, the workflow should also verify the service provider before any attempt. “I have not received any response” and “an operation was not performed” are two separate issues. This tiny difference stops numerous duplicate payments, bookings, ticket purchases, and notifications.
Authentication Can Pass While Authorization Is Wrong
Once a connector has obtained the access token, the groups usually consider the security effort complete. The token is valid, and the integration can continue. The proper question to ask is which operations should the token be allowed to perform. For example, a service account might have rights to modify all invoices within the provider. The process was supposed to modify only the invoices of a specific tenant/region/legal entity. In case of incorrect tenant ID mapping, a minor data problem turns into a cross-customer issue because of broad credentials. There are also permissions drifts. Scopes change, certificates expire, refresh tokens expire, adminstrative consent is withdrawn, and a connector which utilized an employee’s credentials is stopped due to a changed role. The problem is embarrassing because the integration could still be working for some users and not others. A generic availability check won’t spot this situation. Credential monitoring would detect the affected tenant and scope. Reauthorization requires knowledge of the owner and a means of recovering from the outage. The integration itself also requires a mechanism for backfilling changes made during downtime. But most important, the permission check must be near the business action. Access to an endpoint doesn’t mean access to everything that endpoint publishes.
Rate Limits Create Quiet Gaps
Rate-limit problems rarely appear during a calm demo. They show up during a data import, month-end close, campaign launch, provider outage, or recovery job. The queue grows, workers speed up, and the integration sends a burst of requests. Some calls succeed. Others receive 429 Too Many Requests. If the code treats the whole batch as successful, a slice of data disappears. If it retries immediately, the provider may keep throttling the client and the backlog gets worse. [Microsoft Graph’s throttling guidance](https://learn.microsoft.com/graph/throttling) tells clients to respect the Retry-After header. When that header is absent, it recommends exponential backoff. The practical lesson is simple: the provider, not the client, controls the recovery pace. A reliable worker limits concurrency, adds jitter to retries, and caps the number of attempts. It also decides what happens to work that reaches the cap. Does it move to a dead-letter queue? Does a person review it? Can a nightly comparison find the missing records later? Without that last step, a temporary rate limit becomes a permanent data gap.
A Useful Log Tells the Customer’s Story
During an incident, logs often say things like “request completed,” “job retried,” and “message acknowledged.” These entries help developers trace code. They do not answer the questions support and finance are asking. Was the customer charged? Which plan should be active? Did the refund reach the payment provider? Is it safe to run the action again? The best integration logs connect technical events to one business operation. A correlation ID should follow the workflow across services. The record should include the tenant, business object, event version, attempted action, and final known state. Sensitive values should be redacted instead of copied into every log line. This does not mean storing every payload forever. More logs can create privacy and security problems of their own. The goal is a compact trail that lets someone reconstruct what happened without opening five dashboards and comparing timestamps by hand. I would also monitor contradictions directly. Paid account with no entitlement. Refund with no ledger entry. Completed order with no shipment. These checks find the failures customers notice, even when request-level metrics look excellent.
Recovery Deserves a Design Session
Most integration plans spend plenty of time on the happy path and a few minutes on error handling. That order should be reversed for steps that change money, access, or customer data. Before launch, decide what happens after partial success. Can the completed step be reversed? Should the job retry automatically? Does the system need to verify the provider’s state first? At what point should a person take over? The solution varies based on the action taken. An update to a marketing tag can generally be retried. However, giving a refund needs more thought. The deletion of customer data could need proof, authorization, and an unmodifiable log. Reconciliation needs to be part of the architecture. An audit schedule could compare the true source with the copies of it in the systems. It might spot paid accounts without access rights, CRM leads without subscriptions, or invoices that are not in the books. Safe differences should be fixed automatically. The others should enter a review queue with sufficient context for manual evaluation. This beats finding out about them after the customer raises the issue weeks later.
Questions Worth Answering Before Launch
A long integration checklist can become another document nobody reads. I would rather get clear answers to a smaller set of uncomfortable questions: • What specific outcome with the customer indicates that the workflow is completed? • Which system owns each key field? • How are users, organizations, tenants, and subscriptions correlated? • What happens when the same event comes twice or comes late? • Which actions may be retried without issues? • What data does the connector have access to? Which changes it is supposed to make? • How will the team detect drifting records? • Who is responsible for the problem if three vendors are good but the workflow itself is bad? These questions transform abstract integration efforts into concrete tests. In the subscription model, the important principle might be: every valid payment must correspond to one tenant with the right plan and access to the products. This principle can lead to a test, a notification, and a fix effort. “Make sure that billing and CRM are in sync” is impossible. This principle also reveals who owns what. Somebody needs to answer questions about who verifies the payment, who provides access, how long this process will take, and how support needs to act if there is a conflict.
When a Connector Becomes Product Infrastructure
Many integrations start small. A webhook updates a CRM field. A scheduled job copies invoices. A support tool looks up account details. The code may live in a utility folder because nobody expects it to become critical. Then customers depend on it. An integration becomes part of the product once it takes over one or more of onboarding, billing, access, reporting, or support. Versioning, testing, monitoring, security review, recovery mechanisms, and ownership become required at that point. Changes to the provider API are just like changes to the internal API. It is generally at this point that people start thinking about creating a separate integration layer, workflow engine, event store, or canonical data model. It could also be the right time to bring a [SaaS application development company](https://spaculus.com/services/saas-application-development-company/) that has experience with multi-tenancy and production recovery into the mix. The important thing to ask from any potential development partner in such cases is not how many providers’ APIs they have integrated. The important thing to find out is how they deal with duplicate events, partial successes, boundaries of tenants, delayed messages, and reconciliation.
The API Was Working. The Promise Was Not
Return to the customer from the opening incident. The card charge was real. The successful responses were real. So were the locked features and missing email. The team did not need another endpoint. It needed one shared definition of a completed upgrade, one way to match the customer across systems, and one check that compared payment with product access. It also needed an owner who could repair the mismatch without guessing whether the charge should be repeated. That is the difference between connecting software and operating a reliable integration. APIs move requests. The surrounding workflow has to preserve identity, meaning, order, authority, and evidence. When those pieces are designed together, a green dashboard starts to mean something useful. Until then, “the API worked” is only the beginning of the investigation.
Chris Mcdonald has been the lead news writer at complete connection. His passion for helping people in all aspects of online marketing flows through in the expert industry coverage he provides. Chris is also an author of tech blog Area19delegate. He likes spending his time with family, studying martial arts and plucking fat bass guitar strings.