What is context engineering? Designing the information an AI agent needs

Prompt engineering became the popular name for improving an AI response by changing the instructions. That remains useful. A clear role, task, format, example, and constraint can make a single-turn request much more reliable. But an agent is not a single-turn request. It works through a changing environment. Each turn may involve a new tool result, a permission check, a retrieved document, a partial failure, or a decision made earlier in the run. The relevant engineering problem is therefore bigger than the prompt.

Anthropic describes context as the set of tokens included when a model samples a response. Context engineering is the work of curating that set so the model has the highest-signal information for the behavior you want. That includes system instructions, tools, Model Context Protocol connections, external data, message history, and the state generated during the agent loop. The practical aim is not maximum context. It is useful context at the moment a decision is made.

I think of context as the agent's working environment. A person can ask a finance analyst to review an invoice because the analyst knows the company's policy, has access to the ledger, can see the purchase order, and understands the approval chain. A model has none of that unless the application supplies it. If the context contains the invoice but not the tolerance policy, the model may extract accurately and still recommend the wrong action.

This framing also clarifies why prompt changes sometimes appear to fix a problem and then fail in production. A developer may add more instructions to compensate for missing data, stale retrieval, ambiguous tools, or poorly represented state. The prompt grows longer while the underlying context remains unreliable. Context engineering asks a more useful question: what information should be present, in what structure, at what step, from which source, and with what permission?

Language models can process only a finite amount of context at a time, even when the advertised window is large. More tokens also mean more cost, latency, and opportunities for irrelevant or contradictory instructions. Attention is a practical budget. A system that sends an entire knowledge base on every turn may technically fit and still perform worse than one that retrieves a handful of relevant passages and supplies a clear record of why they were selected.

The useful optimization target is signal density. Keep the current goal, constraints, permissions, and next decision obvious. Include evidence that can change the decision. Exclude old conversation that no longer affects the task, repeated policy text, decorative prose, and documents that merely contain the same keywords. Summaries can reduce volume, but a summary should preserve uncertainty, source identity, unresolved questions, and the facts needed to audit a later action.

Context also has a temporal dimension. A product price, account balance, calendar slot, access permission, or legal policy can become stale. Static instructions may be loaded at the start of a run, while volatile data should be retrieved immediately before the action that depends on it. The agent should know when a result was fetched and whether it is safe to reuse. A cached answer is not automatically a current fact.

A good design makes omission deliberate. If a field is not needed for the next decision, do not expose it. This reduces privacy risk as well as noise. The principle is especially important for workplace systems that handle employee, customer, financial, health, or confidential business data. The smallest context that supports a correct decision is usually easier to secure, cheaper to run, and simpler to explain to a reviewer.

Retrieval-augmented generation is often explained as searching documents and putting the results into a prompt. That is only the beginning. The retrieval policy decides what counts as relevant, which source outranks another, how much text is returned, how freshness is handled, and what happens when the search finds nothing or finds conflicting answers. Those are context-engineering decisions because they shape the evidence available to the model.

Anthropic's recent agent guidance highlights a shift from loading all relevant data up front to just-in-time context. Instead of placing every document in the context at the start, an agent can maintain lightweight identifiers such as file paths, stored queries, or web links and load the detailed content through tools when the task reaches the point where it matters. This can preserve a smaller working context and let the system choose what to inspect next.

Just-in-time retrieval is not an excuse for unbounded browsing. The agent needs a source policy. Decide which domains, repositories, databases, or document versions are authoritative. Require citations or record identifiers for important claims. Distinguish an empty result from a failed search. Limit the number of retrieval calls and record the query and returned sources. When two sources disagree, expose the disagreement or escalate rather than blending them into a confident sentence.

Chunking and ranking also affect behavior. A passage that contains a policy exception without the definition it qualifies may be technically relevant and practically dangerous. Preserve headings, dates, table relationships, and document identity where possible. Test retrieval with questions whose answer depends on a small detail, a negative condition, or a recent update. The hard examples reveal whether the system finds evidence or merely finds words.

A model can select only from the tools the application exposes and describes. Tool definitions are therefore context, not incidental metadata. A vague tool name or a large overlapping toolset makes the next-action decision harder. An interface such as update_record(record_id, fields, reason) communicates less authority than separate operations for propose_change and apply_approved_change, even if both eventually call the same API.

Each tool should explain its purpose, inputs, output shape, permission requirements, common failure states, and whether the action is reversible. The schema should reject invalid arguments before they reach the underlying system. The description should tell the model when not to use the tool. These details give the model a smaller and more meaningful choice set while giving engineers a clear place to enforce authorization.

Read and write actions should be distinct. A research agent may retrieve a customer policy, but it should not automatically change the customer's account. A support agent may draft a refund response, but a refund tool should enforce limits and approval. A coding agent may run tests in a sandbox before it can modify a protected branch. The context tells the model what is possible; application code decides what is allowed.

Tool results should return structured evidence. Include status, identifiers, timestamps, relevant fields, and explicit errors. A blank response forces the model to infer what happened. A structured not_found result supports a clear follow-up. A permission_denied result supports escalation. A transient failure supports a bounded retry. The shape of the result is part of how the next turn understands reality.

A long transcript is not a reliable database. It is easy for important facts to be buried among reasoning, duplicated tool output, and corrections. Store the state that drives decisions in explicit fields: current goal, completed steps, extracted values, source identifiers, approval status, retry count, unresolved questions, and next permitted actions. Pass a concise view of that state into the model and keep the full trace available for audit.

Structured state also makes recovery possible. If a browser call fails after a purchase was submitted, the agent should be able to check the transaction identifier rather than blindly retrying. If a human approves a proposed record update, that approval should be represented as an event with an actor and time, not as a sentence the model might misread later. If a required field is missing, the state should say so explicitly and route the task to a question or handoff.

Memory is a separate design choice. Working memory describes the current run. Long-term memory persists between runs and can hold preferences, summaries, or recurring facts. Persistent memory may be useful, but it increases privacy, deletion, correction, and access-control obligations. Do not save every conversation by default. Define what is remembered, who can retrieve it, how it is updated, and when it expires.

Summarization can keep a run within budget, but evaluate the summary like any other component. Does it preserve negations, numbers, names, uncertainty, source links, and pending actions? Does it distinguish a recommendation from a completed action? A short summary that drops one exception can be worse than a longer state object with explicit fields. Compression is valuable only when it retains the information that changes the next decision.

Anthropic describes two common instruction failures. One is brittle over-specification: a prompt hardcodes so many branches and exact phrasings that it becomes difficult to maintain. The other is vague under-specification: the agent is told to be helpful and use judgment without being told what action, evidence, or boundary matters. Good instructions sit between those extremes. They state the goal, the important rules, the decision boundaries, and the expected output without trying to encode the entire application in prose.

Use the application for deterministic rules. If a refund above a threshold needs approval, enforce that in code and mention the rule in the instructions. If a date must be in a particular format, validate it in a schema. If an employee cannot view a record, enforce permission at the data layer. Instructions can explain why and guide the model, but they should not be the only barrier between a model and a consequential action.

Examples are useful when the desired behavior is difficult to describe. Include representative good outputs, a borderline case, and a failure case. Show how the agent should cite a source, ask for missing information, escalate a conflict, or stop after a tool failure. Do not fill the prompt with dozens of examples that repeat the same easy pattern. Choose examples that teach a boundary or a judgment the evaluator cares about.

Treat instructions as versioned product behavior. Record which version ran, test changes against a fixed evaluation set, and review real failures after deployment. A prompt that improves one example and harms three others is not an improvement. The team should be able to answer what changed, why it changed, which cases were tested, and whether the new context increased cost or latency.

A final answer can look good for the wrong reason. The agent may have guessed correctly, used an outdated source, called an unnecessary tool, or skipped a required approval. Evaluation should inspect the path as well as the result. Track whether the right evidence was retrieved, whether the source was current, whether the selected tool matched the task, whether the arguments were valid, and whether the stopping condition was satisfied.

Build an evaluation set from real work. Include common cases, incomplete requests, ambiguous language, conflicting documents, permission boundaries, stale records, long inputs, adversarial instructions, and external content that tries to redirect the agent. Label the expected evidence and acceptable action, not only a gold-standard sentence. For many workflows, the correct result is a clarifying question or a human handoff.

Measure task success, factual support, tool-choice accuracy, unnecessary retrieval, latency, token and API cost, escalation quality, and failure recovery. Review false confidence separately from visible failure. An agent that says it cannot answer may be safe; an agent that invents a policy citation may be dangerous even if the prose sounds excellent. Quality needs a risk-weighted view, not a single average score.

Change one context component at a time where possible. Compare a shorter retrieval set, a different state representation, a clearer tool description, or a revised instruction against the baseline. Keep a regression suite because improvements in one task can create failures in another. Context engineering is iterative system design. It is not a one-time prompt rewrite followed by hope.

Start by writing the decision the agent must make, not the prompt you want it to answer. What is the next action? What evidence would justify it? What permissions apply? What would make the decision unsafe? What does success look like in the source system? This turns a vague agent project into a context contract. The contract gives the team something concrete to test.

Next, inventory the possible context sources: instructions, policies, user profile, permissions, conversation, database records, documents, tool results, and prior events. Mark each source as stable or volatile, authoritative or advisory, sensitive or ordinary, and required or optional. This simple inventory often reveals that a proposed agent has no trusted source for a key decision or has been given more data than it needs.

Design the smallest useful first turn. Give the agent the goal, constraints, available tools, permission context, and only the evidence needed to choose the first action. Let it retrieve details through narrow tools as the workflow progresses. Add structured state after each action. Keep the trace. Then test the failure paths before adding more autonomy. A narrow system with visible limitations is easier to improve than a broad system whose behavior cannot be explained.

Finally, connect context to ownership. The subject-matter expert owns policy meaning. The data owner owns source quality. Security owns access and threat controls. Operations owns workflow outcomes. Engineering owns orchestration and observability. The product owner decides whether the measured benefit is worth the cost and risk. Context engineering succeeds when these responsibilities meet in a tested workflow, not when one person writes a clever system prompt.