An AI agent is not just a chatbot with more tools. The key difference is that the model can participate in deciding what happens next until a goal is complete, blocked, or handed back to a person.
A useful mental model is:
goal
↓
observe current state
↓
model chooses next action
↓
use tool / produce intermediate result
↓
observe new state
↓
continue, stop, or escalate
OpenAI's current agent guidance describes agents as systems where an LLM manages workflow execution, dynamically selects tools, recognizes completion, and can halt or transfer control when necessary. Anthropic makes a similar distinction between predefined workflows and agents whose behavior is model-directed.
That means the agent is best understood as a runtime loop around a model, not as the model itself.
The five parts of a production agent
Most useful agents contain five layers:
1. Model
2. Instructions / context
3. Tools
4. State / memory
5. Runtime controls and verification
The model provides reasoning and language understanding. The runtime gives that intelligence access to real systems while limiting what it may do.
If any one of these layers is weak, the agent becomes unreliable.
1. The model decides, but should not own everything
The model is usually responsible for decisions such as:
Which tool should I call?
Do I need more information?
What does this tool result imply?
Is the task complete?
Should I ask the user a question?
But critical application responsibilities should remain outside the model:
authorization
business invariants
rate limits
budgets
idempotency
approval state
canonical data
For example, a model may decide that a refund looks appropriate. The billing backend still decides whether that refund is actually allowed.
2. Instructions define the agent's job
An agent needs a clear operating contract.
Useful instructions describe:
- objective
- scope
- preferred behavior
- available tools
- important constraints
- when to stop
- when to ask for help
Weak instruction:
You are a helpful billing agent.
Stronger instruction:
Resolve billing questions using approved account and policy tools.
Never invent live account state.
Do not execute refunds above the automatic threshold.
Escalate when account ownership cannot be verified.
The goal is not a giant prompt. It is a clear responsibility boundary.
Context is the agent's working view of the world
The model cannot reason about information it never receives.
Context may contain:
user request
recent conversation
system instructions
retrieved documents
memory
structured application state
tool results
More context is not always better.
Long histories can contain stale decisions, duplicate information, irrelevant tool output, or contradictory state. Modern models support large context windows, but context quality still matters.
A strong runtime continuously asks:
> What is the smallest high-signal context needed for the next decision?
3. Tools connect the model to reality
Without tools, an agent mostly reasons over text.
Tools let it retrieve current data or perform actions.
Typical read tools:
search_docs(query)
get_customer(id)
get_invoice(id)
query_metrics(service)
read_file(path)
Typical write tools:
create_ticket(...)
update_record(...)
send_message(...)
create_refund(...)
Typical execution tools:
run_tests()
execute_command()
browse_page()
A tool should expose a business capability, not unlimited access to the underlying infrastructure.
Good tool design makes agents smarter
Models perform better when tools are clear and distinct.
Bad tool set:
get_data(type, source, mode, filters, options, ...)
Better:
get_invoice(invoice_id)
get_subscription(subscription_id)
search_refund_policy(query)
Anthropic's engineering guidance on tools emphasizes that tool definitions and evaluation strongly affect agent performance. A powerful model can still behave poorly when tools are ambiguous or overlapping.
Good tools should have:
- descriptive names
- narrow responsibilities
- structured input
- structured output
- predictable errors
Read tools and write tools have different risk
A read-only agent can inspect systems.
A write-enabled agent can change them.
That distinction should exist in architecture, not only in prompting.
Example permissions:
research agent → read only
support agent → read + low-risk ticket updates
finance agent → refund proposal, approval required for execution
The model should never gain a new capability merely because it asks nicely.
The core agent loop
At its simplest, an agent runtime behaves like:
while not finished:
response = model(context, tools)
if response is final:
return response
if response requests tool:
validate call
result = execute tool
append result to state
Production systems add more machinery around this loop:
timeouts
budgets
cancellation
approvals
persistent state
retry classification
tracing
postcondition checks
That surrounding machinery is what turns a demo into a reliable service.
Observe → decide → act → verify
A strong agent cycle can be described as four phases.
Observe
Gather relevant state.
user input
tool results
retrieved knowledge
previous run state
Decide
The model chooses the next action.
call tool
ask question
delegate
finish
Act
The runtime performs the allowed action.
Verify
Check whether the result actually moved the task toward completion.
This final step is frequently missing.
Verification matters more than confidence
A model saying:
The tests now pass.
is not evidence.
A coding agent should run the tests.
A billing agent saying:
The refund was issued.
should be backed by authoritative payment state.
Production agents should prefer observable postconditions over model confidence.
State is different from conversation history
Conversation text is useful, but critical workflow state should often be structured.
Instead of forcing the model to infer:
We already verified invoice 42 three messages ago.
store:
{
"invoice_id": "INV-42",
"ownership_verified": true,
"refund_eligible": true,
"approval_status": "pending"
}
This makes retries, resumes, debugging, and human handoffs far more reliable.
Durable state matters for long-running agents
An agent that runs for 30 seconds can sometimes live inside one request.
An agent that may run for 30 minutes, wait for approval, or resume tomorrow needs persistence.
Durable state may include:
run ID
current status
pending tool call
completed steps
approval state
artifacts
budget consumed
If a worker crashes, the runtime should know where the task stopped.
Memory is not the same as state
State answers:
What is happening in this task right now?
Memory answers:
What information from previous interactions might help future tasks?
Examples of memory:
user prefers concise reports
team uses PostgreSQL
customer requested Hindi responses
previous migration decision
Examples of workflow state:
current deployment ID
pending approval
invoice being investigated
Mixing the two creates stale and confusing agents.
Short-term vs long-term memory
Current agent-memory frameworks commonly distinguish:
Short-term memory
Thread/session-scoped information such as recent conversation and working state.
Long-term memory
Information that persists across separate sessions.
Long-term memory can further be divided into:
semantic → facts and preferences
episodic → past experiences/events
procedural → learned instructions or procedures
Not every agent needs all three.
Retrieval is not memory
RAG retrieves external knowledge:
What does the company policy say?
Memory retrieves information about previous interactions or learned user/application state:
Which reporting format did this user choose last time?
Both may use vector search, but their lifecycle and trust rules are different.
Agents should use live tools for live state
If the user asks:
Is invoice INV-42 paid?
query the billing system.
Do not rely on:
- old conversation memory
- yesterday's vector index
- model knowledge
A useful source-of-truth rule is:
knowledge → RAG
live state → tools
previous preferences/experience → memory
hard rules → code
Planning can be explicit or implicit
Some agents plan several steps before acting.
Others decide one step at a time.
Explicit planning can help when tasks are complex:
Goal: investigate latency regression
Plan:
1. compare recent deployments
2. inspect p95 latency by service
3. examine traces for slow dependency
4. validate hypothesis
But planning itself should not become bureaucracy.
For simple tasks, one-step reasoning is often enough.
Replanning is one reason agents are useful
A deterministic workflow may say:
check A → check B → check C
An agent can adapt:
check A
→ A shows database saturation
→ skip irrelevant branch B
→ inspect database metrics
That ability to change path based on evidence is where model-controlled execution creates value.
Retries need failure classification
Agents should not retry every failure the same way.
Example:
provider timeout → retry may help
rate limit → wait/backoff
invalid tool arguments → model may repair
permission denied → retrying is pointless
unknown mutation outcome → verify state first
Treat failures as typed runtime events, not generic strings.
Side effects need idempotency
Suppose an agent calls:
create_refund()
The payment service succeeds, but the network response is lost.
The agent retries.
Without idempotency, two refunds may be created.
For important mutations, use stable operation IDs or idempotency keys and verify ambiguous outcomes before retrying.
Agents need budgets
An autonomous loop can consume resources indefinitely unless the runtime enforces limits.
Typical budgets:
max model calls
max tool calls
max tokens
max wall-clock time
max cost
max parallel workers
A model may decide when it believes the task is done. The runtime still decides how long it is allowed to keep trying.
Cancellation is part of correctness
If a user presses Stop, the system should stop real work.
That may require:
cancel model request
cancel child tasks
signal tool process
prevent new mutations
persist cancelled state
A UI that merely stops displaying tokens while the agent keeps running is not cancellation.
Human approval belongs in the loop for high-impact actions
Some actions should remain reviewable:
large refunds
production deployments
external legal communication
permission changes
data deletion
The agent can gather evidence and prepare the exact action.
A human approves the specific mutation.
This keeps the agent useful without granting unlimited authority.
Handoff is different from failure
A strong agent should know when to return control.
Useful handoff reasons:
missing critical information
repeated tool failure
policy ambiguity
high-risk action
user requests human
The handoff should preserve what the agent already learned so the human does not restart the investigation.
Single-agent vs multi-agent
Start with one agent whenever possible.
Multi-agent architectures are useful when work has real independent boundaries:
parallel research
separate domains
separate permissions
context isolation
ownership transfer
Anthropic's production research system uses a lead agent that decomposes research into independent parallel subagents. Their published experience also highlights higher token use and coordination complexity.
Multi-agent should solve a real bottleneck, not decorate an architecture diagram.
Observability is the agent's debugger
A useful trace should show:
model call
tool selected
tool arguments
result class
latency
state transition
approval
final outcome
You should be able to answer:
Why did the agent call this tool?
What did the tool return?
Why did it retry?
What made it stop?
Without this, production failures become prompt archaeology.
Evals are how you know the agent is improving
Agents are probabilistic.
Do not judge quality from a few demos.
Build representative tasks and measure:
task success
correct tool selection
argument correctness
side-effect correctness
latency
cost
human correction
For agentic tasks, evaluate outcomes and critical safety invariants rather than enforcing one exact harmless tool sequence.
A complete example: support billing agent
User:
I was charged twice. Refund the duplicate payment.
A well-designed run could be:
1. Read authenticated customer identity
2. Get recent invoices/payments
3. Identify likely duplicate
4. Retrieve current refund policy
5. Validate refund eligibility in code
6. If high value → request approval
7. Execute idempotent refund
8. Verify refund state
9. Explain confirmed outcome
The model decides which evidence it needs and how to communicate.
The application still owns authorization, limits, idempotency, and source-of-truth state.
When you do not need an agent
Do not build an agent if the workflow is known and stable.
Example:
upload invoice
→ extract fields
→ validate schema
→ save
That is a workflow.
Use model intelligence inside deterministic software where appropriate.
Agentic autonomy is valuable only when adaptive decision-making improves the task enough to justify extra cost, latency, security, and testing complexity.
Production checklist
Before shipping an agent, verify:
- The goal has a clear completion condition
- The model controls only decisions that benefit from model reasoning
- Tools are narrow and structured
- Live facts come from authoritative tools
- Critical workflow state is structured and durable where needed
- Memory is separate from task state
- Read/write permissions are explicit
- High-risk actions have approval gates
- Important mutations are idempotent
- Ambiguous outcomes have verification paths
- Postconditions are checked before claiming success
- Retries are failure-aware and bounded
- Token/tool/time/cost budgets exist
- Cancellation propagates to real work
- Human handoff preserves context
- Traces expose important decisions and actions
- Representative evals exist before major model/prompt changes
Final takeaway
An AI agent is best thought of as a controlled decision loop around a model.
The model interprets context and chooses the next action. Tools connect it to real systems. Structured state preserves progress. Memory carries useful information across interactions. Runtime controls constrain authority, cost, retries, and duration. Verification proves whether important actions actually happened.
The strongest architecture is not the one that gives the model the most freedom.
> Give the model freedom where reasoning helps, and keep deterministic control where correctness, security, and business invariants must always hold.
That is how agent intelligence becomes reliable software.
References and further reading
Discussion (0)