How AI agents work, from request to verified action

The easiest mistake is to treat an AI agent as a special kind of language model. It is better understood as a software system built around a model. The model interprets the situation and proposes a next action. The application decides what information the model receives, which tools it may select, whether an action needs approval, what result comes back, and when the run must stop. Change that surrounding system and the same model can behave like a chatbot, a research assistant, or an operations agent.

OpenAI describes an agent through three foundations: a model, tools, and instructions. Anthropic makes a useful additional distinction. A workflow follows predefined code paths, while an agent dynamically directs its own process and tool use. In practice there is a spectrum. A customer-support system can have a fixed outer workflow, deterministic policy checks, and one agentic step that interprets an unusual request. Calling the entire product autonomous would hide the controls doing most of the reliability work.

I use a practical test. If the model can only return text to the user, it is an assistant or chatbot. If the model can choose actions that change workflow state through tools, it has agentic behavior. That does not mean unlimited authority. A well-designed agent is usually bounded: it has a narrow job, a small toolset, explicit permissions, observable state, and a point at which it asks a person to decide.

Words such as reasoning, planning, and memory are engineering shorthand. They describe observable functions, not proof that a model thinks like a human. The model generates likely useful actions from its training and current context. The application then turns a selected action into conventional software execution. Keeping that separation clear makes agents easier to evaluate and makes failures less mysterious.

Imagine a small agency receives supplier invoices by email. The goal is not merely to summarize an attachment. The useful outcome is to identify the supplier, extract the amount and due date, compare the invoice with an approved purchase order, prepare a record in the accounting system, and flag anything suspicious for review. That is a multi-step job involving unstructured documents and structured business rules.

The run begins when the email system passes the message and attachment to the agent. The application adds the agency's invoice policy, the user's permissions, and tool descriptions. The model first chooses a document-extraction tool. It receives structured fields rather than relying on its own recollection of the attachment. It then calls a purchase-order lookup with the extracted order number. If no match exists, the correct next action is escalation, not improvisation.

Suppose the order exists but the invoice is three percent higher. A policy may allow a two-percent tolerance. This comparison should be deterministic code because the rule is crisp. The agent receives the failed policy result and drafts a concise review note explaining the discrepancy. It does not post the payable record. A human can reject the invoice, approve an exception, or correct the extracted value.

If every check passes, the agent prepares a structured accounting entry. Creating that entry may still require approval during the pilot. After approval, a tool writes the record and returns an identifier. The agent verifies the identifier by reading the saved record, then marks the email processed and reports completion. A credible success condition is not 'the model said done.' It is evidence from the system of record that the intended entry exists with the expected fields.

This example shows why the surrounding architecture matters. Document extraction, database retrieval, policy comparison, approval, record creation, and verification are separate operations. The model chooses among them and handles ambiguous language. Deterministic software enforces rules and permissions. The human owns exceptions. The agent is valuable because it coordinates the pieces, not because it replaces every piece with generated text.

At each turn, a model receives a context window containing some combination of instructions, conversation, tool definitions, retrieved information, and prior tool results. It cannot automatically inspect your database, browser, email, or earlier runs. The application must deliberately retrieve and present relevant information. Too little context causes uninformed decisions. Too much irrelevant context increases cost and can bury the instruction that matters.

Working memory is usually a state object for the current run: the goal, completed steps, extracted values, tool results, approvals, and remaining work. This state is more reliable when stored in structured fields than when represented only as a long transcript. The system can validate an invoice amount as a number, a due date as a date, and an approval as a recorded event. Structure reduces the chance that a later model call misreads its own narrative.

Long-term memory persists between runs. It may store a user's preferred report format, recurring supplier details, or a summary of an earlier case. It is not automatically beneficial. Old memories can be wrong, retrieved for the wrong person, or retained longer than expected. Before adding persistent memory, define what is stored, why it improves the task, who can read it, how users correct it, and when it is deleted.

Retrieval is often mistaken for memory. A search tool can fetch current policy documents when needed without permanently teaching the model or copying every document into every request. That pattern is usually preferable for changing business knowledge because the source remains identifiable and updateable. The agent should return citations or record identifiers so a reviewer can inspect the evidence that shaped its action.

A tool is a controlled interface between the model and ordinary software. Its definition tells the model what the tool does and which arguments it accepts. When the model selects the tool, the application validates those arguments, checks permissions, runs the underlying function, and returns the result. A search tool may call an index. An email tool may create a draft. A database tool may retrieve one authorized record. The model proposes; code mediates.

Good tool design is narrow and explicit. A tool named manage_customer_data with a free-form instruction is difficult to secure and evaluate. Separate tools such as get_customer, draft_address_change, and submit_address_change make authority visible. The submission tool can require an approval token that the model cannot invent. The application can log exactly which customer and fields changed.

Read and write tools deserve different treatment. Reading public documentation is low impact. Sending a message under a person's name, deleting a file, altering production data, or making a purchase can create real consequences. OpenAI's guide recommends risk-rating tools and adding checks before high-risk actions. A useful default is automatic reads, constrained drafts, and explicit approval for consequential writes.

Tool results should be designed for machines and people. Structured status codes help the agent distinguish not found from permission denied or temporary failure. Human-readable explanations help reviewers understand the event. Returning an empty string or a vague 'failed' encourages the model to guess. Clear errors allow bounded retries or a clean handoff.

Every agent run needs several stopping conditions. The positive condition is task-specific evidence of success: a saved record identifier, a passing test suite, a confirmed booking, or an answered question with cited sources. Generic confidence is not enough. The application should be able to inspect the evidence without trusting the model's assertion that it succeeded.

Negative conditions prevent runaway behavior. Set a maximum number of model turns and tool calls, a wall-clock limit, a budget ceiling, and retry limits for each external service. Detect repeated action-and-result pairs because they indicate a loop. Stop when required information is missing, permissions are insufficient, or two authoritative sources conflict. A handoff with a concise state summary is a successful outcome when automation cannot safely continue.

Retries need policy. A temporary network error may justify an exponential backoff. An invalid account number should not be retried with invented alternatives. A declined approval ends the action. An authentication failure goes to an administrator. Encoding these distinctions in software prevents a language model from treating every obstacle as an invitation to be creative.

Cost is also a stopping concern. Agents may call models repeatedly, retrieve large documents, invoke paid APIs, or create downstream review work. Track cost per completed outcome, not just token spend. A cheaper model that causes more retries and human correction may be more expensive overall. A strong evaluation set lets a team compare architectures using completion quality, latency, and total operating cost.

No single prompt can secure an agent. Instructions help shape behavior, but authorization belongs in code. A model told never to issue a large refund should still be physically unable to call the refund tool above an enforced limit. Data access should use the current user's permissions. Sensitive actions should require fresh approval. Logs should record inputs, selected tools, validated arguments, outputs, and final state without exposing unnecessary private data.

Prompt injection is especially important when an agent reads external content. A web page, email, or document can contain text telling the agent to ignore its task or reveal information. External content should be treated as untrusted data, not system instruction. Limit which tools are available during retrieval, separate data from instructions, validate outputs, and require approval before an untrusted source can influence a consequential action.

Guardrails can include input checks, relevance classifiers, content-safety checks, personally identifiable information filters, tool-specific authorization, output validation, and human review. OpenAI describes these as layered defenses. NIST's risk framework adds organizational discipline: define ownership, map risks in context, measure them with evidence, and manage them throughout deployment rather than only before launch.

The least glamorous control is often the most useful: begin in read-only or shadow mode. Let the agent recommend actions while people continue making the decisions. Collect failures, ambiguous cases, and disagreement data. Turn those examples into evaluations. Grant narrowly scoped write authority only after the system meets a threshold, and preserve a rapid way to disable it.

Anthropic recommends starting with the simplest solution because agentic systems often exchange latency and cost for flexibility. That is excellent advice. If a process is a stable sequence of known steps, ordinary automation will be easier to test and cheaper to run. A form submission that always creates the same record does not need a model choosing what to do next.

Agents become useful when the path depends on unstructured input, exceptions, or context that would create an unmaintainable rule tree. Insurance correspondence, complex support requests, research across heterogeneous documents, and coding tasks can fit this pattern. Even then, keep deterministic islands for calculations, permissions, policy limits, and final validation.

Start with one agent and a small set of distinct tools. Multi-agent architecture adds handoffs, duplicated context, more failure surfaces, and harder attribution. It is justified when specialized roles need genuinely different instructions or tool access, or when parallel work creates measurable value. It is not a quality upgrade by itself. A single well-instrumented agent often outperforms a group whose responsibilities overlap.

My final test is operational: can the team define success, assemble representative examples, observe every consequential action, contain failure, and name the person who owns the workflow? If not, the organization is not ready to automate that job with an agent. Improve the process first. Agents amplify the clarity or confusion already present in the workflow they inherit.