Why integrations fail: APIs, data and retry patterns

A system must know who owns data, when to send a change and what counts as confirmation. A POST is not enough when the network fails after a write.

Map objects and events: create, update, cancel, retry and manual correction.

Define required fields, date formats, identifiers, schema versions and incompatible changes. Document limits, timeouts and partial responses separately.

The contract must be testable. A JSON snippet is not a test suite for real failures and edge cases.

A retry is unavoidable when a request succeeds but its response is lost. Idempotency keys and an operation log prevent duplicate orders or payments.

Use a queue, processing status and a way to retry only the failed step for long-running work.

Engineers need logs; operators need status, next retry and an action they can take. Latency, error rate and queue age reveal degradation before a customer complaint.

That makes the integration an owned product capability rather than an invisible script.

An integration fails when two systems own the same field, interpret a status differently or cannot distinguish a retry from a new operation. Map objects, events and sources of truth before adding queues.

Define who may correct data and how that correction reaches the other system. Manual edits without a log create an unexplained discrepancy.

The network may fail after a successful write. Use an idempotency key and persist the attempt result. Batch exchange needs a cursor or checkpoint to resume from the last confirmed step.

Never blindly retry payments, orders or notifications. Separate technical retry, manual resend and compensation when the provider already changed state.

Connect the input event, queue, provider request, database write and user notification in incident review. Correlation IDs and structured logs make the chain searchable.

After fixing the cause, add a regression test and a metric that reveals recurrence before a customer does.

Collect correlation ID, request time, retries, provider response and local state. The sequence matters more than one error because it shows where confirmation was lost.

If logs are insufficient, do not enable full request bodies blindly. Add safe fields and secret redaction so diagnosis does not create a second incident.

Timeouts, temporary outages and rate limits may use backoff. Validation, permission and stale-ID errors need data correction or a manual decision.

One retry policy for every code creates request storms and duplicates. Make the policy part of the integration contract and expose it in metrics.

Add a regression test, an alert for the earlier signal and a support runbook. Verify whether an unfinished operation can be safely resumed.

The purpose of a postmortem is not blame. It is to make the next failure shorter, clearer and cheaper.

Follow operation IDs through the request, queue message, partner response and local record. Compare timing, contract version and payload at each transition.

This reveals where data was lost, duplicated or accepted with different semantics. Repeating the request rarely fixes that class of failure.

Keep timeout, duplicate webhook, partial response and master-data change as executable scenarios. Run them whenever an adapter or API version changes.

An integration is resilient when the team can prove a fix did not reintroduce an earlier mismatch.

Define a retry key and repeat result for every operation. An operator should see that an order already exists instead of pressing Submit again. The rule must match API, queue and recovery UI.

A status field can mean accepted, paid or handed over in different systems. Map transitions and test impossible combinations. Valid JSON does not guarantee a valid business operation.

After recovery record timeline, affected operations, detection signal and the change that prevents recurrence. “Restarted the service” is not a learning. A short report improves architecture and the runbook.