What is prompt chaining, and when does it earn its extra step?

A single prompt asks a model to understand a task, plan it, perform it, check it, and present the result in one pass. That can be perfectly adequate for simple work. It becomes fragile when the request contains several different kinds of reasoning. Prompt chaining separates those jobs into a sequence of model calls, usually coordinated by application code.

Anthropic describes prompt chaining as decomposing a task into a sequence of steps, with each model call processing the previous output. The important detail is the gate: the application can run a programmatic check between calls to see whether the work is still on track. A chain is therefore more than putting several prompts in a row. It is a workflow with contracts and decision points.

The sequence is normally known in advance. A marketing workflow might extract a brief, propose three angles, check them against brand rules, write the selected draft, and produce channel variants. A support workflow might classify a request, retrieve policy, draft a response, check for unsupported claims, and route a sensitive case to a person. The model performs steps; the application owns the route.

This makes chaining different from an autonomous agent. An agent may choose which tool to call and when to stop based on the current state. A chain usually has a predetermined shape, even if some steps use a model. That predictability is useful when the process needs repeatable approvals, clear testing, or a known audit trail.

The strongest candidates have subtasks with different success criteria. Extracting fields from a document is judged by completeness and schema validity. Reviewing a draft is judged by a rubric. Generating a polished version is judged by usefulness and style. Keeping these jobs separate gives each prompt a clearer target and gives the system a better place to catch failure.

A chain is also useful when intermediate output is valuable to a human. A reviewer may want to see the extracted facts, the risk flags, or the outline before approving the final answer. The intermediate artifacts make the process explainable and let a person correct the state before a costly or consequential step.

Do not chain simply because more model calls sound more sophisticated. If the task is a short classification, one structured call may be easier to operate. If the subtasks are tightly coupled and every intermediate representation loses important context, a chain may add friction without improving the result. Start with a one-call baseline and compare it to the proposed chain.

Ask four questions: Can I name the steps? Can I define what each step must return? Can I check the output before continuing? Can the next step use a compact, stable representation? If the answer to most is yes, chaining is worth testing. If not, improve the task definition before adding orchestration.

Draft, review, revise is useful when quality is easier to judge after a first attempt. The reviewer should not be asked only whether the draft is 'good.' Give it a rubric: missing claims, unsupported claims, audience fit, required sections, tone, and prohibited language. The revision step should receive the draft and the actionable findings, not an unbounded transcript.

Extract, validate, transform separates seeing from doing. The extractor returns a schema. A deterministic gate checks fields, types, ranges, and required evidence. Only then does a later call produce a summary, database payload, or recommendation. This pattern is valuable when downstream systems need predictable inputs and when a malformed extraction must not silently become a polished error.

Plan, execute, verify is helpful when an action has a visible result. The plan can identify files, queries, or steps. Execution is limited to the approved scope. Verification checks the final state instead of trusting the model's claim. For production code, verification might run tests; for a report, it might check required sections and source coverage; for a record update, it might inspect the system of record.

Classify, route, specialize sends a request to a focused prompt or workflow. The classifier should expose uncertainty and have a fallback for ambiguous cases. Routing should not be the only security boundary: the specialist must still enforce its own permissions and input rules. A wrong route can be costly, so include confusing examples in the evaluation set.

A useful step contract says what it receives, what it must return, what it must not do, and what happens when it cannot complete the step. Prefer structured output when the next stage is code or another model. Use stable field names, explicit null behavior, and a place for uncertainty or missing evidence.

Pass only the context needed for the next decision. Anthropic's work on context engineering emphasizes that context is finite and that irrelevant material can dilute attention. A chain creates an opportunity to summarize or compress intermediate state, but compression can also remove a qualification. Preserve source references and important uncertainty when reducing context.

Make authority clear. If a step receives a retrieved policy and a user request, the policy is evidence and the system rules determine what actions are permitted. If a reviewer identifies a problem, the revision step should treat that finding as a required correction, not as another suggestion to ignore. Clear labels reduce conflicts among instructions, data, and previous output.

Version each prompt and schema. A chain can fail when one stage changes its field name or interpretation and the next stage still expects the old contract. Log the versions alongside each run. When a change improves one stage but harms the whole workflow, the trace should show where the difference entered.

A gate is a check between steps. It can be deterministic, such as schema validation, a regex, a database lookup, a unit test, a policy rule, or a required citation. It can be model-based for qualities that need language judgment, provided the rubric is clear and calibrated. It can be human when the consequence or nuance warrants review.

A gate should have a failure path. The application might retry with a correction, ask the user for missing information, send the work to a human, or stop with a useful explanation. Retrying the same prompt without changing the input or instruction often just spends more money to reproduce the same error.

Keep gates narrow. A reviewer asked to judge grammar, factual support, privacy, tone, and business strategy all at once may produce an impressive but unstable score. Separate checks when the downstream action differs. A privacy violation should stop the workflow even if the draft is otherwise excellent.

Some gates should be outside the language model. Authentication, authorization, spending limits, file paths, database constraints, and irreversible actions belong in application code and service permissions. A prompt can tell the model not to send an email; the mail service still needs a separate rule that controls whether sending is allowed.

Chaining can improve quality while increasing latency and cost. Count model calls, tokens, retries, search operations, and human review time. Measure time to useful completion rather than only time to first response. A three-step workflow that prevents a costly error may be a good trade; a five-step workflow that adds no measurable value is not.

Create a test set with ordinary tasks, edge cases, incomplete inputs, conflicting instructions, and cases where the correct behavior is to stop. Compare the chain with the one-call baseline. Evaluate each step and the end state. An intermediate score can look strong while a small loss of information causes the final answer to fail.

Track error propagation. If extraction misses a number, the reviewer may approve a flawed summary because the number is absent. If routing chooses the wrong specialist, later steps can be excellent and still irrelevant. Record the state entering each step, the gate result, retries, and whether a human corrected it.

Use production corrections as new tests. When a user edits a draft or rejects a route, preserve the anonymized case and the reason. A chain should become more predictable because its failure modes are being converted into requirements, examples, and regression checks.

Imagine a customer asks, 'Can I get a refund for the extra seats we paid for last month?' A single model call might draft a plausible answer, but the answer depends on account identity, plan terms, dates, usage, and approval limits. A chain makes those dependencies visible.

Step one classifies the request and extracts the account identifier, time period, product, and requested outcome. If the identifier is missing, the chain asks a clarifying question. Step two retrieves the applicable policy and account record. A permission gate confirms that the user and support role may see the data.

Step three calculates or proposes the eligible amount using code, not model arithmetic alone. Step four drafts a response that separates confirmed facts from the proposed action. A policy gate checks that the reply does not promise an approval the agent cannot grant. Step five either sends the draft to an authorized reviewer or presents it to the customer according to the product's approval design.

This workflow is longer than 'ask the model to answer the customer.' It is also easier to test. Each step exposes a state, each action has a boundary, and a failure can be routed without pretending that a fluent sentence proves the refund is valid.

When a chained workflow produces a bad result, start with the first state that became wrong. The final answer is usually where the problem becomes visible, not where it began. Compare the original input, the exact prompt version, the model response, the parsed fields, the gate result, and the context passed to the next step. This trace tells you whether the failure came from interpretation, extraction, validation, routing, context loss, or an action outside the model.

Suppose a research chain produces a confident summary that cites the wrong number. The extraction step may have copied the number correctly but dropped the source location. The synthesis step then had no way to distinguish the figure from a nearby table. The fix is not necessarily a stronger final prompt. It may be to require every extracted claim to carry a source identifier, page or section reference, date, and confidence state, then make the gate reject claims without that evidence.

Retries should be diagnostic. A retry can use a smaller input, a clearer schema, a different model, a missing-information question, or a human correction. Record which intervention was used and whether it changed the outcome. Blindly repeating the same call can make reliability look better in a small demo while multiplying cost in production. A useful retry budget has a maximum number of attempts and a clear destination when the budget is exhausted.

Context loss deserves its own check. If step one produces a long narrative and step two receives only a summary, compare the summary against the fields the next decision actually needs. Preserve negative findings, uncertainty, exclusions, and provenance; these are often the first details removed by an enthusiastic compression step. If the chain needs the full source, pass a reference that the application can retrieve rather than hoping a model-generated paraphrase retains every qualification.

Finally, distinguish a model error from a workflow error. A model may misunderstand an ambiguous instruction, but the application may also have allowed an unvalidated value to reach a sensitive action. The model can be improved with examples and a better contract. The workflow must be improved with permissions, deterministic checks, limits, and an escalation path. Treating every failure as a prompt-writing problem leaves the actual control boundary unprotected.

Before shipping a chain, assemble a set of cases that represent the work rather than the marketing demo. Include ordinary inputs, incomplete inputs, ambiguous requests, conflicting instructions, long documents, unusual formatting, and examples where the correct result is to ask for clarification or stop. For each case, write the expected properties of the outcome. You do not need to prescribe every word, but you do need a way to decide whether the result is acceptable.

Run the one-call baseline and the chain on the same cases. Record end-to-end task success, factual or field accuracy, unsupported claims, escalation rate, latency, model cost, retry count, and human correction time. Then inspect the steps separately. A chain may improve final quality by increasing review time, or reduce model cost while increasing support burden. Those are product decisions, not details to hide behind a quality score.

Use a simple release rule. For example, the chain must meet the same or better task-success rate as the baseline, reduce a named failure category, keep the 95th-percentile latency below the workflow's tolerance, and stay below a cost ceiling per accepted outcome. High-risk actions should have a separate approval threshold. If the evidence is mixed, launch the chain in shadow mode or with a human approval step instead of granting broader autonomy.

Keep the evaluation set alive after launch. Every material correction becomes an anonymized regression case, with the reason the output failed and the stage where the failure entered. Re-run the set when you change a prompt, model, parser, retrieval method, or gate. This is how a chain becomes an engineered workflow instead of a sequence of prompts that happened to work once.