Adding more agents does not automatically make an AI system smarter.
It can just as easily create more prompts, more context, more latency, more tool calls, more security boundaries, and more failure modes.
Multi-agent architecture becomes valuable when separation itself solves a real problem: independent work can happen in parallel, different specialists need different context or permissions, one coordinator needs to delegate bounded tasks, or a conversation genuinely belongs to different expert agents at different times.
The enterprise question is therefore not:
> “How many agents should we build?”
It is:
> Which responsibilities benefit from being isolated, delegated, or parallelized—and which should remain in one agent or normal deterministic software?
That distinction prevents a useful agent platform from turning into a distributed system made entirely of prompts.
Start with one agent unless separation earns its cost
A single capable agent with well-designed tools is often the best starting point.
Keep one agent when:
- the task needs one coherent context
- subtasks depend heavily on each other
- the same tools and permissions apply throughout
- latency matters more than parallelism
- one model can solve the task reliably
- delegation does not create a clear ownership boundary
Introduce additional agents when you can name the benefit.
Good reasons include:
parallel independent research
specialized domain instructions
separate tool permissions
context isolation
different model requirements
clear conversational handoff
independent workspace execution
“Enterprise AI should be multi-agent” is not a reason.
What actually makes a system multi-agent?
A useful definition is simple:
Agent A
↓
coordinates, delegates, or transfers work
↓
Agent B / C / D
Each agent usually has some independent combination of:
instructions
model
context
tools
permissions
state
completion criteria
The important architectural question is who owns the task at each moment.
That leads to two core patterns.
Pattern 1 — Manager / agents as tools
One central agent keeps ownership of the user-facing task and calls specialists for bounded work.
User
↓
Manager Agent
├── Research Agent
├── Database Agent
├── Security Agent
└── Financial Agent
↓
Manager synthesizes final result
↓
User
OpenAI's Agents SDK calls this the agents-as-tools pattern.
The specialist behaves like a callable capability. It returns its result to the manager rather than taking over the conversation.
Use this when
- one agent should own the final answer
- global policy should remain centralized
- specialist work is a bounded subtask
- outputs from several specialists need synthesis
- the user should not need to understand internal delegation
Example
A cloud-architecture agent might ask:
security specialist → review IAM model
cost specialist → estimate infrastructure trade-offs
database specialist → review data architecture
The manager combines those reports into one recommendation.
Pattern 2 — Handoffs
A routing or triage agent transfers ownership to a specialist.
User
↓
Triage Agent
├── billing request → Billing Agent
├── technical issue → Support Agent
└── contract issue → Legal Ops Agent
After the handoff, the selected specialist becomes the active agent.
Use this when
- one specialist should directly continue the conversation
- each domain needs focused instructions
- each specialist has substantially different tools
- responsibility clearly changes after classification
A handoff is not just “call another model.” It changes who owns the next turn.
Manager and handoff are complementary
Enterprise systems often combine both.
For example:
Triage Agent
↓ handoff
Technical Support Agent
├── calls Log Analyst as tool
├── calls Database Analyst as tool
└── calls Documentation Researcher as tool
↓
responds to user
The support specialist owns the conversation, but it can still delegate narrow analysis to other agents.
This is usually easier to reason about than building a peer-to-peer swarm where every agent can call every other agent.
Avoid unrestricted agent-to-agent graphs
It is tempting to build:
A ↔ B ↔ C ↔ D ↔ E
where any agent can invoke any other agent.
That quickly creates problems:
- cycles
- duplicated work
- unclear ownership
- unpredictable cost
- hard-to-debug traces
- permission leakage
- agents repeatedly delegating the same task
Prefer an explicit topology.
Examples:
manager → specialists
triage → one domain agent
orchestrator → workers → orchestrator
The relationship graph should be understandable without asking the model to explain it.
Orchestrator-worker is ideal for dynamic decomposition
Some tasks cannot be split into fixed branches ahead of time.
An orchestrator can inspect the task and create workers dynamically.
Goal
↓
Orchestrator
↓
decides decomposition
↓
┌──────────┬──────────┬──────────┐
Worker A Worker B Worker C
└──────────┴──────────┴──────────┘
↓
Orchestrator
↓
synthesis
This works well for:
- large research tasks
- repository migrations
- due-diligence investigations
- incident analysis
- broad document review
Anthropic's production research-agent work found multi-agent systems particularly useful for breadth-first research where independent directions could be explored in parallel.
The same work also highlights the cost: multi-agent systems can consume far more tokens than a single-agent flow.
Parallelism is useful when the task earns it.
Delegation needs a contract
Do not delegate like this:
"Analyze the database."
A worker needs a bounded assignment.
A strong delegation package contains:
objective
scope
inputs
allowed tools
permissions
expected output
completion criteria
budget
Example:
Objective:
Find all database changes required by the tenant-ID migration.
Scope:
backend/db and migration files only.
Do not:
modify files or inspect frontend code.
Return:
- affected file
- table/query
- required change
- migration risk
Stop when:
all schema and query references in scope have been inspected.
Clear contracts reduce overlap and improve synthesis quality.
Specialists should be actually specialized
Creating five agents with almost identical prompts and toolsets is not useful specialization.
A specialist should differ meaningfully in at least one of:
instructions
tool access
model
context
permissions
workspace
success criteria
For example:
Security agent
Can inspect code and configuration but cannot modify production resources.
Deployment agent
Can prepare and validate a deployment plan but production execution requires approval.
Billing agent
Can access billing APIs but not unrelated customer documents.
Specialization should reduce the agent's decision surface, not merely change its name.
Context isolation is one of the strongest reasons to use subagents
Large tasks create enormous context.
One agent may accumulate:
source code
database schemas
observability data
security policies
research results
Most of that is irrelevant to any one subtask.
A specialist can work in a clean context containing only what it needs.
Manager context
↓
minimal task packet
↓
Specialist clean context
↓
structured result
↓
Manager context
This prevents the manager from paying attention to every intermediate detail.
It also reduces cross-contamination between unrelated tasks.
Do not share the entire conversation automatically
A receiving agent rarely needs every token the previous agent saw.
Pass the minimum required handoff state:
{
"customer_id": "cus_42",
"issue_type": "duplicate_charge",
"invoice_id": "inv_99",
"user_goal": "understand and resolve duplicate payment",
"handoff_reason": "billing specialist required"
}
instead of sending:
150 messages
all previous tool outputs
all unrelated documents
Context minimization improves both efficiency and privacy.
Shared state and private state should be different
In a multi-agent run, not all state should be globally writable.
A useful structure is:
Run-level shared state
- user goal
- authenticated identity
- workflow status
- global budgets
Agent-private state
- temporary notes
- local search results
- specialist reasoning artifacts
Durable business state
- database / external systems
Agents should not silently communicate through mutable global prose if a structured shared field would be clearer.
Agent identity should exist outside the prompt
In enterprise systems, “You are the Billing Agent” is not enough identity.
The runtime should know:
agent_id
application_id
tenant/user identity
tool policy
credential scope
run_id
When the agent calls a tool, the downstream service should know which user and which agent are acting.
This enables real authorization and auditing.
Different agents should have different permissions
A strong multi-agent system does not give every specialist the union of all tools.
Example:
Research Agent
- web search
- document search
- no writes
CRM Agent
- read CRM
- update notes
- no billing access
Billing Agent
- read invoices
- propose refund
- execute approved refund
Code Agent
- repository workspace
- sandboxed shell
- no production credentials
If a specialist is compromised or confused, its blast radius remains bounded.
Authentication and authorization must survive delegation
A handoff must never accidentally reset security context.
For every delegated action, preserve:
who is the user?
which tenant?
what permissions?
what approval state?
what operation scope?
The receiving agent should not gain broader access simply because another agent delegated to it.
Authorization belongs in deterministic application code and downstream services—not in the handoff text.
Multi-agent systems need global budgets
Parallel workers can multiply spend quickly.
Suppose:
manager spawns 6 workers
workers each call 10 tools
3 workers spawn additional specialists
A small user request can become a large distributed run.
Set global limits such as:
max_agents_per_run
max_parallel_workers
max_delegation_depth
max_model_calls
max_tool_calls
max_tokens
max_cost
max_wall_clock_time
Also give workers local budgets so one bad branch cannot consume the whole run.
Prevent delegation loops
Without controls, systems can do this:
Agent A → Agent B → Agent C → Agent A
or repeatedly send the same unresolved task between specialists.
Useful defenses include:
- maximum delegation depth
- visited-agent/task signatures
- one owner per work item
- explicit handoff reason
- repeated-delegation detection
The runtime should stop obvious cycles instead of relying on models to notice them.
Parallel workers need conflict rules
Parallelism is easy for read-only analysis.
It becomes harder when workers mutate shared state.
Example:
Worker A edits auth.ts
Worker B edits auth.ts
Now you need conflict resolution.
Safer patterns include:
- workers are read-only and manager applies changes
- workers own disjoint files/resources
- each worker has an isolated workspace/branch
- deterministic merge/validation happens afterward
Do not let “parallel agent” secretly mean “concurrent unsynchronized writers.”
Durable runs matter for enterprise workflows
Long-running multi-agent tasks should survive:
process restart
worker crash
provider timeout
human approval delay
scheduled resume
Persist enough state to reconstruct the run:
{
"run_id": "run_847",
"status": "waiting_for_workers",
"active_tasks": ["security_review", "database_review"],
"completed_tasks": ["api_inventory"],
"budget_remaining": 0.43,
"approval_state": null
}
Do not rely on one process's in-memory conversation objects for a workflow that may run for hours.
Worker failures should not always kill the whole run
Different worker tasks may have different criticality.
Example:
3 research workers
- 2 succeed
- 1 source is unavailable
The orchestrator may still be able to produce a result with a clear limitation.
But if the failed worker was the only security validator for a production change, the run should stop.
Classify worker dependencies:
required
optional
best-effort
This makes failure behavior explicit.
Structured worker outputs make synthesis easier
Do not ask every specialist to return a 5,000-word essay.
Use a contract appropriate to the task.
For example:
{
"findings": [],
"risks": [],
"evidence": [],
"open_questions": [],
"confidence": "medium"
}
The manager can synthesize structured evidence much more reliably than extracting facts from several long narratives.
Multi-agent memory needs ownership
If several agents can write memory, define who owns what.
Possible namespaces:
/user/{user_id}
/project/{project_id}
/agent/{agent_id}
/org/{tenant_id}
/run/{run_id}
A researcher should not be able to rewrite organization policy because it encountered a sentence on the web.
Shared memory must have stronger provenance and write controls than agent-private scratch state.
Prompt injection risk multiplies with delegation
Every new agent, tool, external document, and handoff is another trust boundary.
A malicious document may influence one specialist, which then returns a poisoned recommendation to the manager.
Security design should therefore preserve provenance:
worker says X
based on source Y
source Y is untrusted external content
The manager should not treat every subagent output as trusted policy merely because another internal agent produced it.
Use humans at high-impact boundaries
Enterprise multi-agent systems often handle tasks where some actions should remain supervised.
Useful checkpoints include:
production deployment
large financial action
external legal communication
sensitive data export
permission escalation
irreversible deletion
The workflow should present the proposed action, supporting evidence, and expected effect before approval.
Observability must show the delegation tree
A normal API trace is not enough.
You need to see:
Run
├─ Manager
│ ├─ delegate → Research Agent
│ │ ├─ web_search
│ │ └─ final specialist result
│ ├─ delegate → Security Agent
│ │ ├─ read_file
│ │ └─ final specialist result
│ └─ synthesize
└─ final output
Track:
- agent/model used
- delegation reason
- input/output metadata
- tool calls
- latency
- token/cost usage
- retries
- approvals
- failure class
- completion state
This is essential for debugging why a supposedly simple request became slow or expensive.
Evaluate specialists and coordination separately
Multi-agent evals need more than final-answer quality.
Routing accuracy
Did the correct specialist receive the task?
Delegation quality
Was the delegated scope clear and complete?
Worker task success
Did each specialist complete its assignment correctly?
Coordination efficiency
Did multiple workers duplicate the same work?
Handoff correctness
Was enough context transferred without leaking unnecessary information?
Tool authorization
Did each agent stay within its allowed capabilities?
End-to-end outcome
Did the combined system solve the user's goal?
A final answer may look correct even if the run wasted five unnecessary agents to produce it.
Compare multi-agent against a single-agent baseline
This is one of the most important tests.
Before keeping a multi-agent architecture, compare it with:
one strong model
+
well-designed tools
+
clear context
Measure:
task success
latency
token usage
cost
failure rate
operational complexity
If the multi-agent version is only slightly better but three times more expensive and much harder to debug, it may not be the right architecture.
Where multi-agent systems shine
Good fits include:
Broad research
Independent search directions can run in parallel.
Large codebases
Specialists can inspect separate modules or concerns in isolated workspaces.
Enterprise support
Different domain agents can have tightly scoped tools and policies.
Cross-functional analysis
Security, finance, legal, and technical specialists can provide independent evidence for one decision.
Long, decomposable workflows
An orchestrator can dynamically split a large goal into independent work packages.
Where they are usually unnecessary
Avoid multi-agent design when:
- one agent already performs well
- all subtasks need the same shared context
- the workflow is deterministic
- each “specialist” has the same tools and instructions
- ultra-low latency is critical
- the task cannot be parallelized
- the coordination cost exceeds the quality improvement
Sometimes the best “multi-agent architecture” is no multi-agent architecture at all.
A practical enterprise architecture
User / Application
│
▼
Auth + Tenant Context
│
▼
Agent Gateway
│
▼
Triage / Manager Agent
│ │
┌─────────┼─────────────┼──────────┐
▼ ▼ ▼ ▼
Research Billing Security Code
Agent Agent Agent Agent
│ │ │ │
▼ ▼ ▼ ▼
scoped scoped scoped isolated
tools tools tools workspace
└─────────┬─────────────┬──────────┘
▼
Structured Results
│
▼
Manager / Workflow
│
policy + approval gates
│
▼
Final Action/Answer
Around the entire system sit:
durable run state
budgets
identity and authorization
tracing
evals
memory policy
sandboxing
Those platform layers matter more than the number of agent boxes in the diagram.
Production checklist
Before shipping a multi-agent system, verify:
- Every additional agent has a concrete reason to exist
- Ownership after each handoff is unambiguous
- Manager-vs-handoff behavior is deliberate
- Delegation contracts define scope and completion
- Agent-to-agent topology cannot loop freely
- Delegation depth is bounded
- Global and per-worker budgets exist
- Agent identities exist outside prompts
- Permissions differ by agent role
- User/tenant authorization survives handoffs
- Shared and private state are separated
- Parallel workers cannot corrupt shared state
- Long-running runs are durable/resumable
- Worker failure criticality is explicit
- Specialist outputs are structured where useful
- Memory writes have ownership and provenance
- Delegation trees appear in traces
- Evals test routing, delegation, and coordination
- A single-agent baseline has been measured
Final takeaway
The best enterprise multi-agent systems are not swarms of models talking endlessly to each other.
They are structured organizations of responsibility.
Use one agent when one context and one capability set are enough.
Use managers when one component should remain accountable for the final result.
Use handoffs when a specialist should take ownership.
Use orchestrator-worker patterns when subtasks emerge dynamically and can genuinely be separated.
Then surround the agent graph with the same engineering discipline you would use for any distributed production system: identity, authorization, durable state, bounded resources, failure recovery, observability, security, and tests.
> Split agents by responsibility only when the boundary improves quality, safety, parallelism, or context—not because a diagram with more boxes looks more intelligent.
References and further reading
Discussion (0)