APIs, Events, Data, State, and Distributed Dependencies

Mission

• Measurable outcome: By the end, you can review a distributed flow and identify contract, state, consistency, retry, and ownership risks with a test or decision for each.
• Prerequisites: Chapter 2.1 request tracing.
• Work product: An interface-contract and dependency register.
• Time: 60–75 minutes.

Before you read: Predict → Commit → Connect

A client times out after sending a payment. It retries. The bank approved the first request, but the client never received the response. Predict the safest system behavior and the state needed to produce it. Commit before reading. Connect the scenario to the authoritative-state question from Chapter 2.1.

Distributed systems turn boundaries into behavior

An API is an interface contract for requesting behavior or data. An event records that something occurred and may be consumed asynchronously. Data has meaning, ownership, lineage, quality, and retention, not just a location. State is information whose prior value affects current behavior. A distributed dependency exists when successful behavior depends on another component, network, team, vendor, or clock outside the current unit’s immediate control.

HTTP and OpenAPI can standardize parts of a contract, but a useful program view includes semantics: identity, authorization, schema, versioning, timeout, retry, idempotency, error behavior, rate limit, ordering, and support ownership. Two systems can both conform syntactically and still disagree about whether “approved” means authorized, captured, or settled.

Accessible diagram label: A synchronous payment command changes authoritative state and emits an event consumed independently by ledger, fraud, and reconciliation systems.

Synchronous calls simplify immediate reasoning but couple availability and latency. Asynchronous events decouple timing but introduce lag, duplicate delivery, ordering, schema evolution, and replay questions. Neither is universally better. The TPM asks whether the chosen pattern supports the outcome and whether its consequences appear in plans and evidence.

For the prediction scenario, a common design is to use a stable idempotency key so repeated requests represent one logical operation. The exact solution belongs to engineering. The TPM must ensure the key survives every relevant boundary, duplicate behavior is specified, and verification covers timeouts before and after side effects.

Accessible diagram label: A payment state model includes an explicit unknown state after timeout rather than treating uncertainty as failure and risking a duplicate side effect.

Five program-level questions expose distributed risk:

1. Contract compatibility: Can producers and consumers change independently? What backward or forward compatibility is promised?
2. Delivery semantics: Can a message be absent, delayed, duplicated, reordered, or replayed? What behavior is safe?
3. State authority: Which system decides current truth, and how are replicas or derived views reconciled?
4. Failure containment: Does one slow dependency exhaust threads, queues, connections, or operator attention elsewhere?
5. Ownership continuity: Who monitors, supports, and evolves the interface across organizations and time zones?

Avoid absolute shorthand such as “exactly once.” A system may approximate an exactly-once business effect through deduplication and transactional design, while networks and brokers still retry. Ask engineers to define the guarantee’s scope and failure assumptions.

Also distinguish transport success from business completion. An API can return successfully before an asynchronous obligation settles, and an event broker can acknowledge delivery before a consumer applies it. Name the completion boundary that matters to the outcome.

At Meridian, the ledger team plans a schema change before all regional consumers upgrade. The issue is not merely a missed date. It is contract compatibility across independent deployments. The TPM maps producers, consumers, versions, traffic sequence, rollback compatibility, and test evidence, then drives a decision on schema evolution.

Decision rights: Who owns what?

Service and data owners define and implement contracts. Architects may set organization-wide patterns. Security and privacy authorities govern access and retention. SRE or platform teams may define reliability constraints. The TPM owns cross-team visibility: consumer inventory, compatibility milestones, decisions, end-to-end evidence, and escalation when no one owns an interface.

The TPM should not prescribe a queue or database from shallow familiarity. The TPM can require that alternatives address observed constraints and failure modes.

I do

I review Meridian’s ledger event. I capture producer, seven consumers, field semantics, schema owner, retention, ordering key, duplicate behavior, and replay process. Two consumers interpret an absent region field differently. Rather than declare an implementation, I convene owners, set a compatibility deadline, and require consumer-contract tests plus a replay rehearsal before traffic expansion.

We do

Helios emits an answer_generated event before an agent approves the answer. Analytics treats it as a resolved case; compliance treats it as customer-visible communication. Together separate facts, states, and events. Propose clearer event semantics and identify replay risk.

You do

Select one API or event that crosses teams. Build the register below. Interview one producer and one consumer separately. Compare their definitions of success, error, retry, and ownership. Convert each contradiction into a decision or risk with a due date.

Pause & Recall

From Chapter 2.1, recall the seven system-reading prompts. Which ones become most important for asynchronous behavior? From Chapter 1.2, explain how an interface mismatch can make every component green while the program outcome fails.

Production lens

Observe interface health, not only service health. Track contract-test failures, consumer lag, dead-letter volume, duplicate business effects, reconciliation differences, and version distribution where relevant. Ensure on-call teams can trace one logical operation across retries and events.

Workplace artifact: interface register

Interface / business meaning:
Producer and accountable owner:
Consumers and owners:
Protocol / schema / version:
Authentication and authorization:
Authoritative state:
Timeout, retry, idempotency, duplicate behavior:
Ordering, lag, replay, retention:
Error contract and degraded behavior:
Compatibility promise and rollout sequence:
Observability / trace identifier:
Test and reconciliation evidence:
Support path / change approval:
Open decision and residual risk:

Chapter compression

Contracts carry semantics, not only syntax. Events trade timing decoupling for lag, ordering, duplicate, replay, and evolution concerns. State needs an authority. TPMs integrate ownership, compatibility, rollout, failure behavior, and evidence while engineers own the design.

Retrieval deck

• Q: What does idempotency protect? A: Repeating one logical request without repeating an unintended business effect.
• Q: Why is an event not automatically truth? A: It records an occurrence; authoritative current state and event semantics may differ.
• Q: What does asynchronous delivery add? A: Lag, duplicate, ordering, replay, and schema-evolution considerations.
• Q: Why inventory consumers? A: A producer change can break unknown or independently deployed dependencies.
• Q: What should “exactly once” trigger? A: Questions about scope, assumptions, and the mechanism producing one business effect.

Spaced review

• Now: Explain why a timeout is not necessarily a failed business operation.
• 1 day: Recreate the payment state model without notes.
• 3 days: Trace one duplicate or reordered event through two consumers.
• 7 days: Trace a replay through all known consumers and update compatibility risks.
• 14 days: Audit whether the named interface owner still accepts ownership.

Sources and further study

• IETF RFC 9110, HTTP Semantics
• OpenAPI Initiative, OpenAPI Specification
• AsyncAPI Initiative, AsyncAPI Specification
• Amazon Builders’ Library, Making retries safe with idempotent APIs