Orchestration Patterns Before Frameworks

Architecture is the smallest explicit control system that meets the task, safety, and reliability requirements.

Mission

By the end, you can translate a task into an inspectable control graph; choose sequential, routing, parallel, review, or planner-executor patterns; and reject framework or multi-agent complexity that has not earned its cost.

Prerequisite: Modules 1-4 and a working copy of the offline lab package. Build artifact: an orchestration decision record for the research and support capstones. Time: 70-90 minutes.

Before you read: Predict → Commit → Connect

A research agent sometimes misses one source. A teammate proposes a larger model, an agent framework, three specialist agents, shared memory, and a judge agent.

Before reading, rank those five changes by the evidence you would require. Which failure does each change address? What new failures does each introduce? Commit to the simplest first experiment.

Begin with the control graph

A framework is packaged machinery. An architecture is the set of decisions that determines what happens, in what order, under whose authority, with which state, and how failure ends. Draw that architecture before choosing machinery.

For every node, write:

• its typed input and output;
• whether code or a model chooses the next edge;
• its timeout, retry, and budget;
• the state it reads and writes;
• whether it can cause an external side effect;
• the evidence it emits;
• its failure destination.

If those answers are vague, a framework makes the vagueness run faster. If they are explicit, you can implement the first version with ordinary functions and replace the runtime later.

Six patterns cover much of the useful space

These patterns compose. A router can select a sequential branch; a pipeline can fan out; a write step can pause for approval. Composition is useful only while the resulting graph remains testable.

The decision tree is not a law. It is a forcing function: every escalation must name the uncertainty it handles and the evidence that simpler control failed.

Deterministic edges before model-selected edges

Use code for facts the system already knows: schema validation, authentication, authorization, budget checks, retry classes, fixed dependencies, and business rules. Use a model where semantic judgment over messy input creates value: classifying an ambiguous request, extracting a candidate structure, selecting relevant evidence, or drafting an answer.

Even then, the model proposes. Trusted code validates the proposal and decides whether an operation is authorized. A model-generated plan is untrusted data, not executable authority.

A useful planner output contains observable artifacts such as steps, dependencies, tool names, typed arguments, expected evidence, and stop conditions. It does not require storing or exposing hidden chain-of-thought. Debug the graph and its events, not private reasoning text.

The complexity ledger

Every added branch, model call, agent, queue, or framework feature has a tax. Record it before adoption.

An architecture change earns adoption when a representative evaluation shows a meaningful improvement after these costs are counted. “It feels more agentic” is not evidence.

A control plane for Northstar

The Northstar support flow needs semantic triage, but refund authority belongs to policy and an approver. The research flow needs search and synthesis, but its path is known. Neither needs a framework to express the first production candidate.

The framework boundary, if one is added, should sit behind these application contracts. Do not let a runtime own business authorization merely because it has an “approval” callback.

I do: identify the research capstone’s real pattern

Run the offline research capstone:

cd courses/ai-agents/labs
python3 -m agent_lab.capstones.research "What protects a tool-using agent?"

Its useful architecture is a sequential pipeline:

1. accept and bound the query;
2. search a synthetic corpus;
3. read the matched records;
4. synthesize only from those records;
5. attach citations and finish.

Search can return several records, but that does not automatically require parallel agents. The corpus is local, the dependency order is known, and the final answer has one evidence contract. A future parallel reader would be justified only if measured latency improved enough to repay join complexity and duplicate work.

The important evidence is visible: query, tool calls, observations, citations, status, budgets, and output. There is no need to capture hidden reasoning.

We do: choose a support pattern

Consider tickets for password resets, refunds, privacy deletion requests, and unknown issues.

Together, make these decisions:

1. A deterministic lookup verifies that the ticket exists.
2. A bounded router proposes a category; code validates it against an allowlist.
3. Password reset produces a draft, not an account change.
4. Refund arguments pass schema and policy checks, then require an exact approval.
5. Privacy deletion routes to a specialist human workflow; the agent does not delete.
6. Unknown or low-confidence cases escalate.

This is a router with guarded branches and a human checkpoint. Adding a second “refund agent” would not create refund authority. It would add another probabilistic boundary that still needs the same controls.

You do: LAB-13 — Architecture before runtime

Run both support paths and the focused tests:

cd courses/ai-agents/labs
python3 -m agent_lab.capstones.support T-1001
python3 -m agent_lab.capstones.support T-1001 --approve-refund
python3 -m unittest \
  tests.test_agent.AgentLoopTests.test_approval_denial_prevents_handler \
  tests.test_agent.AgentLoopTests.test_explicit_allowlist_approval_runs_handler -v

The second path and test exercise a tool-name allowlist in a deterministic simulation. They show where an approval checkpoint sits in the graph; they do not implement the exact, authenticated, one-use approval required by the production design above.

Create a one-page decision record with:

1. the current pattern for each capstone;
2. deterministic and model-selected edges;
3. side-effect and approval boundaries;
4. stop conditions and budgets;
5. the measurable failure that would justify routing, parallelism, planning, a framework, or another agent;
6. one evaluation that could disprove your proposed upgrade.

Done when: another engineer can reproduce the control graph without knowing a framework name, and every proposed complexity increase has a metric and rollback path.

Pause & Recall

1. Why draw node contracts before selecting a framework?
2. When is parallel fan-out valid?
3. What is the difference between a model-selected edge and an authorized action?
4. Name four entries in a complexity ledger.
5. What evidence would justify a planner-executor pattern?

Production lens

Version the control graph, prompts, schemas, policies, tool adapters, and models separately. Put a run ID and version set on every trace. Define concurrency and rate limits at fan-out points. Make joins explicit about “all,” “any,” quorum, timeout, and partial-result behavior.

For each branch, test success, invalid proposal, dependency timeout, retry exhaustion, budget exhaustion, approval denial, cancellation, and recovery. Route confidence is not permission. A route can select the refund workflow while authorization still rejects the refund.

Adopt a framework only after a small proof shows it improves required capabilities—such as durable state, scheduling, tracing, or portability—without weakening application-owned controls. Keep an exit seam so framework objects do not become your only business records.

Chapter compression

• Start from a typed control graph, not a framework catalog.
• Prefer known deterministic edges; use models for bounded semantic uncertainty.
• Sequential, routing, parallel, evaluator-optimizer, planner-executor, and human-checkpoint patterns cover much of the useful space.
• Model plans and routes are proposals; code owns validation and authorization.
• Every new component pays taxes in failure states, security, evaluation, latency, cost, and operations.
• Complexity is justified by representative measured gain, not novelty.

Memory hook: Pattern first; contract each edge; make complexity earn its place.

Retrieval deck

• Q: What should be designed before choosing an agent framework?
  A: The explicit control graph, node contracts, authority boundaries, state, budgets, evidence, and failure paths.
• Q: When should independent work fan out?
  A: When tasks truly do not depend on each other and measured benefit repays join and partial-failure complexity.
• Q: Does a planner authorize its proposed tool calls?
  A: No. Trusted code validates the plan, policy, identity, budget, and arguments before execution.
• Q: What is the acceptance test for more architecture?
  A: Representative evaluations show material net improvement after reliability, security, latency, cost, and operational taxes.

Spaced review

• Now: draw the research capstone as a five-node graph.
• +1 day: reconstruct the six orchestration patterns from memory.
• +3 days: price the complexity tax of parallelizing one step.
• +7 days: replace one model-selected edge in a design with deterministic code and compare it.
• +14 days: audit a framework-based agent and identify which business controls can be extracted behind stable interfaces.

Sources and further study

• Anthropic: Building Effective Agents
• van der Aalst et al.: Workflow Patterns
• Yao et al.: ReAct—Synergizing Reasoning and Acting in Language Models
• NIST Secure Software Development Framework, SP 800-218