Context Engineering—Build the Decision View

Mission

By the end, you can assemble a small, trustworthy context packet, separate instructions from untrusted data, and test whether compression preserved what the task needs.

Prerequisite: Chapter 2.1. Build artifact: a context manifest with provenance and a token budget. Time: 60-75 minutes.

Before you read: Predict → Commit → Connect

An agent has a 100,000-token input limit. It has 80,000 tokens of chat history, policies, search results, tool documentation, and notes. Should you include everything because it fits?

Write three ways “more context” could make the next decision worse.

Context is the model’s temporary decision view

Context is the information made visible to a model for one inference. It can contain instructions, the current task, selected state, tool definitions, retrieved evidence, recent observations, and output requirements.

Context engineering asks: What is the smallest, clearest, properly labeled packet that enables this decision?

It is broader than prompt wording. A beautifully written instruction cannot repair missing evidence, stale state, conflicting documents, an oversized tool catalog, or hostile text placed where instructions belong.

The complete database, entire transcript, and every available tool are not context by default. They are candidate sources.

Six layers in a useful decision packet

Organize context by role and trust:

1. Stable instructions: role, goal interpretation, policy, prohibited behavior, response contract.
2. Current task: the user’s request and authenticated scope.
3. Authoritative state: compact facts owned by application code—status, budgets, approvals, IDs.
4. Available actions: only tools eligible at this step, with clear schemas.
5. Evidence and observations: relevant passages and tool results, explicitly labeled as untrusted content with provenance.
6. Output contract: the exact response or action schema and completion criteria.

Keep these layers distinguishable even if an API represents them differently. Data that says “ignore all previous instructions” remains data.

Instruction precedence is not a security boundary by itself

Platforms define instruction roles or authority. Use them correctly. But a model can still be influenced by hostile lower-authority text. Enforce security outside the model with tool allowlists, authorization, validation, network controls, and approvals.

The context manifest

For each included item, record metadata:

This makes a response explainable without recording private reasoning: “The decision used policy v18, booking snapshot b42, and search result r7.”

A token budget is an attention budget

Even when content fits the technical window, irrelevant volume can bury important details, introduce contradictions, raise cost and latency, and enlarge the prompt-injection surface. Research has shown that models can use information differently depending on where it appears in long context; placement and selection matter.

Allocate a budget by function, not by whatever arrives first. For example:

These numbers are examples, not universal targets. Measure on your models and tasks.

Compress by information class

• Structured state: project fields; do not summarize away exact values.
• Conversation: keep unresolved commitments and a short recent window; store a reviewed summary separately.
• Documents: retrieve relevant passages with surrounding context and citations.
• Tool results: normalize fields; retain raw results by reference for audit.
• Instructions: remove repetition and resolve conflicts at authoring time.
• Tool catalog: expose only actions permitted and useful in the current state.

Summarization is lossy. Test it like any other transformation.

Retrieval-augmented generation, clearly

Retrieval-augmented generation (RAG) selects external information and supplies it to a generator. It is not automatically an agent. A fixed retrieve-then-answer pipeline is a workflow. It becomes agentic when the model may adapt retrieval—rewrite a query, inspect a source, search a different collection, ask a question, or stop.

A reliable retrieval pipeline considers:

• document ingestion and parsing quality;
• chunk boundaries and metadata;
• lexical and semantic retrieval;
• tenant and authorization filters before results reach the model;
• reranking, diversity, and duplicate control;
• source authority, date, and version;
• citation mapping;
• tests for retrieval recall and answer groundedness separately.

Vector similarity is not permission and not truth.

Three context failures that look like model failures

Missing

The answer requires a policy exception that was never retrieved. The model improvises. Fix retrieval coverage or teach the system to abstain; a stronger prompt cannot see absent facts.

Conflicting

Two policy versions disagree, and neither is labeled. Fix source authority and freshness. Do not ask the model to guess which organization policy is valid.

Polluted

A retrieved page contains hostile instructions or irrelevant repeated text. Label it untrusted, minimize it, validate all proposed actions, and consider excluding the source. Prompt injection is a system problem, not merely a prompt-writing problem.

I do: reduce a bloated Northstar packet

Candidate packet:

• 38,000 tokens of chat history;
• the full 200-page travel policy;
• 42 tools across six departments;
• a 9,000-token booking JSON response;
• ten web results;
• current request: “Show Friday options under CAD 900; do not book.”

Construct the decision view:

1. extract authenticated route and constraints into authoritative state;
2. retain the user’s current request and one unresolved preference;
3. retrieve only refundability and change-policy passages from the current policy version;
4. expose search_options and ask_user, not booking or unrelated tools;
5. project booking JSON to the necessary route, ticket class, and policy code;
6. keep relevant web/catalog passages with source, date, and IDs;
7. state an output contract: three evidence-backed options or a clarification, never a write.

The smaller packet is not less capable. It is easier to reason about, cheaper, and safer.

We do: test a summary

A prior conversation contains:

• user prefers Friday after 14:00;
• direct flight preferred but one stop accepted;
• CAD 900 is a hard ceiling;
• user explicitly said “do not book”;
• earlier suggestion YYZ–SFO-17 was rejected because arrival was too late.

A generated summary says: “User wants an affordable Friday flight and prefers direct.”

What was lost? The exact time, allowed fallback, hard price constraint, prohibition on booking, and negative preference. A safe summary test includes questions for every constraint and prohibition. If the summary cannot answer them, it is not a valid replacement.

You do: LAB-5 — Context diet

Create a synthetic 30-item packet containing:

• five trusted instructions;
• five exact state facts;
• ten relevant and irrelevant document passages;
• two conflicting policy versions;
• one hostile passage;
• seven tool descriptions, only two of which are useful.

Your target packet may contain at most 12 items.

1. Assign manifest metadata to every candidate.
2. Select the packet and justify each inclusion.
3. Write five “must answer” probes and three “must refuse” probes.
4. Compare results before and after compression.
5. Check that citations still map to exact source passages.
6. Check that the hostile passage cannot enable an unavailable tool.

Done when: the reduced packet preserves all required answers and prohibitions, rejects the attack, and removes at least half of the candidates.

Pause & Recall

1. What is context engineering’s central question?
2. Name the six layers of a decision packet.
3. Why is “it fits” a poor inclusion rule?
4. How can retrieval fail independently of generation?
5. What must a context manifest record?

Production lens

Log source IDs, versions, selection reasons, and token counts—not entire sensitive content by default. Apply access control during retrieval, not after generation. Redact before external model calls where needed. Define retention for prompts, tool results, caches, traces, and provider-side storage separately.

Build context regression tests. When a prompt, policy, retriever, chunker, model, or tool catalog changes, verify that critical facts remain present, forbidden data remains absent, citations still resolve, and adversarial content cannot expand authority.

Chapter compression

• Context is the temporary decision view, not everything the system knows.
• Separate trusted instructions, authoritative state, tools, and untrusted evidence.
• Budget attention; more tokens can reduce quality and increase risk.
• RAG is retrieval plus generation, not automatically an agent.
• Provenance, freshness, scope, and sensitivity travel with content.
• Compression is lossy and must be tested.

Memory hook: Select, label, budget, verify.

Retrieval deck

• Q: How is context different from a data store?
  A: Context is the selected information visible for one inference; a store contains candidate information.
• Q: Is retrieved text trusted?
  A: No. It carries provenance and authority metadata but remains untrusted content.
• Q: What turns RAG into an agentic pattern?
  A: The model adapts retrieval or the path based on observations.
• Q: What is the safest way to shorten authoritative state?
  A: Deterministically project required fields; do not replace the source of truth with a model summary.

Spaced review

• Now: sketch the six-layer packet.
• +1 day: find three items you would remove from a real prompt.
• +3 days: design one missing, conflicting, and polluted context test.
• +7 days: explain why retrieval quality and answer quality need separate metrics.
• +14 days: rerun your context diet after changing one source version.

Sources and further study

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Lost in the Middle: How Language Models Use Long Contexts
• OpenAI, Conversation State
• OWASP, Prompt Injection Prevention Cheat Sheet