Agentic Workflow Design: Patterns, Orchestration, State, and Reliable Execution

Most failed agent systems do not fail because the model is incapable. They fail because the workflow gives the model the wrong amount of freedom.

One team hard-codes every step and wonders why the system cannot adapt. Another gives an LLM twenty tools and an open-ended goal, then wonders why it loops, chooses the wrong action, burns tokens, or becomes impossible to debug.

Good agentic workflow design sits between those extremes.

The workflow should be deterministic where the process is known and agentic where judgment, exploration, or adaptation is genuinely required. The model should make the decisions that benefit from intelligence; normal software should continue to own the rules that benefit from certainty.

That distinction is the foundation of reliable agentic systems.

What is an agentic workflow?

An agentic workflow is a multi-step process in which one or more LLM-powered components reason, use tools, inspect intermediate results, and help decide what should happen next.

But there are two importantly different ways to build one.

Workflow-driven execution

The application owns the control flow.

Step A
  ↓
Step B
  ↓
validate
  ↓
if condition X → Step C
else            → Step D

The LLM may perform individual steps, but code decides which steps exist and how they connect.

Anthropic uses the term workflow for this style: LLMs and tools are orchestrated through predefined code paths.

Agent-driven execution

The model owns more of the control flow.

Goal
  ↓
LLM inspects state
  ↓
chooses next tool / subtask
  ↓
observes result
  ↓
decides what to do next
  ↓
repeats until done

Anthropic describes agents as systems where the LLM dynamically directs its own process and tool usage. OpenAI's current Agents SDK makes the same architectural distinction from another angle: orchestration can be model-driven, code-driven, or a mixture of both.

Neither style is universally better.

The right question is:

> How much of this workflow actually needs model judgment?

The most useful design rule: deterministic spine, agentic islands

A strong production default is to keep the major business process deterministic while giving the model autonomy inside bounded parts of it.

For example, consider a refund-support workflow.

Bad version:

"Resolve this customer's billing problem. You can use all billing tools."

The model now decides everything: what to inspect, whether the customer qualifies, how much to refund, and whether to execute the refund.

A safer design looks like:

Authenticate customer                 ← deterministic
        ↓
Load invoice                          ← deterministic tool
        ↓
Classify billing issue                ← LLM judgment
        ↓
Retrieve applicable policy            ← deterministic retrieval
        ↓
Explain cause / propose resolution     ← LLM judgment
        ↓
Validate refund eligibility            ← deterministic business rules
        ↓
Request approval if required           ← deterministic policy
        ↓
Execute idempotent refund              ← deterministic tool
        ↓
Generate customer explanation          ← LLM

The model adds flexibility where language and reasoning matter. Code retains control where money, permissions, and invariants matter.

This hybrid shape is often much easier to test and operate than either a fully hard-coded pipeline or a fully autonomous agent.

Start with the task shape, not the framework

Before choosing LangGraph, an Agents SDK, ADK, a visual workflow builder, or your own runtime, draw the task.

Ask:

1. What triggers the workflow?
2. What is the concrete success condition?
3. Which steps are always required?
4. Which decisions are ambiguous enough to need an LLM?
5. Which work can happen in parallel?
6. Which actions have side effects?
7. Which actions require human approval?
8. What state must survive retries or restarts?
9. What should happen when a step fails?
10. How will we know whether the workflow actually succeeded?

Frameworks implement the graph. They do not decide whether your graph is good.

The core agentic workflow patterns

Most useful systems can be built by composing a small number of recurring patterns.

Anthropic's engineering guidance identifies five especially useful patterns: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer. In practice, production systems often combine these with bounded agent loops and human approval checkpoints.

Pattern 1 — Sequential chaining

Use a sequence when the task naturally breaks into ordered stages and each stage makes the next one easier.

Input
  ↓
Extract facts
  ↓
Draft answer
  ↓
Check policy
  ↓
Format output

Example: turning a support ticket into a high-quality response.

raw ticket
   ↓
extract customer intent + entities
   ↓
retrieve account context
   ↓
draft response
   ↓
policy/compliance check
   ↓
final response

Why chaining helps

One giant prompt has to solve every subproblem simultaneously.

A chain can give each step:

• narrower instructions
• smaller context
• a clear input schema
• a clear output schema
• an independent test

Add gates between stages

Do not blindly pass one model's output into the next model.

Validate important boundaries:

extract
  ↓
schema valid?
  ├── no → retry/correct/fail
  └── yes
        ↓
next stage

Use sequential chaining when the order is stable. Do not turn a predictable pipeline into an autonomous planner merely because “agents” sound more advanced.

Pattern 2 — Routing

Routing chooses the right path for the input.

Request
   ↓
Router
   ├── billing workflow
   ├── technical support workflow
   ├── account workflow
   └── general answer

Routing can be deterministic or model-driven.

Use deterministic routing when rules are crisp

if webhook.type == "payment_failed":
    run_payment_failure_flow()

Do not pay an LLM to decide something your program already knows.

Use model-driven routing when classification is semantic

For example:

"I was charged twice after changing my plan and one of the invoices is wrong."

That request may require language understanding to decide whether it belongs to billing, subscriptions, or a combined workflow.

Make route outputs structured

A router should return something like:

{
  "route": "billing_dispute",
  "confidence": 0.93,
  "reason": "duplicate charge after plan change"
}

not a paragraph that another component has to interpret.

Pattern 3 — Parallelization

Run work concurrently when subtasks are independent.

                ┌→ investigate logs ───────┐
Incident ───────┼→ inspect recent deploys ─┼→ synthesize
                ├→ search past incidents ──┤
                └→ check dependencies ─────┘

This can reduce wall-clock time dramatically when each branch can work without waiting for the others.

Good examples include:

• researching independent topics
• checking multiple data sources
• reviewing a change from security, correctness, and performance perspectives
• analyzing separate files/modules
• generating several candidate solutions for later comparison

Do not parallelize dependencies

If task B depends on the result of task A, parallel execution just creates rework or inconsistent state.

Parallel does not mean unlimited

Bound the fan-out:

max_parallel_workers = 4
max_total_subtasks   = 8
max_tool_calls        = 40

Anthropic's production research-agent work shows why this matters: multi-agent systems are powerful on breadth-first, parallelizable tasks, but coordination and token cost increase quickly.

Pattern 4 — Orchestrator and workers

Use this when the subtasks cannot be fully known in advance.

Goal
  ↓
Orchestrator
  ↓
creates plan dynamically
  ↓
 ┌──────────┬──────────┬──────────┐
Worker A   Worker B   Worker C
 └──────────┴──────────┴──────────┘
             ↓
        Orchestrator
             ↓
          synthesize

This is different from ordinary parallelization.

In fixed parallelization, you know the branches at design time.

In orchestrator-worker systems, the model discovers the decomposition from the actual task.

Example: a repository migration.

The orchestrator may inspect the codebase and decide that this particular migration needs:

worker 1 → database schema changes
worker 2 → API compatibility
worker 3 → frontend call sites
worker 4 → test migration

Another repository may require completely different work.

Good delegation needs a contract

Do not send a worker this:

"Research the backend."

Give it:

objective
scope
available tools
expected output
completion criteria
constraints

For example:

Objective: find every call site affected by the auth API change.
Scope: backend/services and shared/auth only.
Do not modify files.
Return: file path, symbol, why affected, migration note.
Stop after all search paths are exhausted.

Clear worker boundaries reduce duplicated work and missing coverage.

Pattern 5 — Evaluator and optimizer

Some tasks improve when one model creates and another critiques.

Generate
   ↓
Evaluate against rubric
   ↓
Pass? ── yes → finish
   │
   no
   ↓
Feedback
   ↓
Revise
   └──────────→ evaluate again

Good fits include:

• writing against a clear editorial rubric
• code review/fix loops
• structured extraction where completeness can be checked
• plans with explicit acceptance criteria
• generated SQL or configuration with validation

The key is that the evaluator needs real criteria.

Bad evaluator instruction:

"Make it better."

Better:

Check:
- all required fields are present
- claims are supported by provided evidence
- output follows schema
- no prohibited action is suggested
- answer directly resolves the user's request

Bound the loop

Never allow open-ended self-improvement.

max_iterations = 3

If the third attempt still fails, stop, escalate, or return a controlled failure.

Pattern 6 — Bounded autonomous loop

Sometimes the task really is open-ended enough that the model should choose the next action repeatedly.

A basic loop looks like:

while not complete:
    observe current state
    choose next action
    execute action
    inspect result
    update state

This is useful for:

• debugging
• research
• coding tasks
• browser/computer work
• exploratory analysis

But the application must define the outer boundary.

At minimum:

max steps
max model calls
max tool calls
max runtime
max tokens
max cost
max writes

The model may decide the path. It should not decide whether the path can run forever.

Pattern 7 — Human approval checkpoints

Human-in-the-loop is not a fallback for bad AI. It is a workflow primitive.

Use it where the cost of a wrong action is higher than the cost of waiting.

Examples:

Agent prepares refund
        ↓
amount > threshold?
        ├── yes → human approval
        └── no  → execute automatically

or:

Agent prepares production deployment
        ↓
show diff + tests + risk summary
        ↓
engineer approves
        ↓
deploy

Approval should happen before the side effect

This sounds obvious, but designs sometimes ask the model to act and then request confirmation afterward.

The workflow should separate:

proposal
approval
execution
verification

Those are four different states.

Model the workflow as explicit state

Multi-step agent systems become much easier to reason about when state is explicit instead of hidden inside chat history.

A workflow state might look like:

{
  "run_id": "run_123",
  "status": "awaiting_approval",
  "goal": "refund duplicate charge",
  "customer_id": "cus_42",
  "invoice_id": "inv_99",
  "classification": "duplicate_charge",
  "proposed_refund": 2499,
  "completed_steps": [
    "load_invoice",
    "classify_issue",
    "validate_policy"
  ],
  "next_step": "approve_refund"
}

Now a restart does not require the model to reconstruct reality from twenty pages of transcript.

Separate durable state from conversational context

Conversation history is useful for language continuity.

Workflow state should contain operational truth.

Do not make this:

"I think we already refunded the invoice earlier in the conversation."

carry the same weight as this:

{
  "refund_operation_id": "rf_abc",
  "status": "succeeded"
}

Treat each step like an API contract

Every important workflow node should have:

input contract
output contract
preconditions
side effects
failure modes
retry policy
completion condition

This is one of the biggest differences between a demo chain and a production workflow.

Example

Step: create_refund

Input:
- invoice_id
- amount
- reason
- idempotency_key

Preconditions:
- invoice belongs to authenticated tenant
- amount <= refundable balance
- approval present when threshold requires it

Success output:
- refund_id
- confirmed amount
- provider status

Failure classes:
- validation failure
- authorization failure
- provider rejected
- timeout / outcome unknown

The model should not need to infer these semantics from a vague tool name.

Side effects change everything

Reading data and changing data should not share the same workflow policy.

A failed read is usually safe to retry.

A failed-looking write may already have succeeded.

Consider:

Agent calls create_order()
        ↓
server commits order
        ↓
network response times out
        ↓
agent sees timeout

If the workflow blindly retries, you may create two orders.

Mutations need idempotency

Give retried operations a stable identity:

idempotency_key = workflow_run + logical_action

A retry should produce the same operation, not another operation.

Unknown outcome is its own state

Do not reduce every tool result to success/failure.

Useful states are:

confirmed_success
confirmed_failure
outcome_unknown

For outcome_unknown, verify first.

timeout
   ↓
check provider by operation ID
   ↓
exists? → continue as success
missing? → safe retry

Retries should depend on the failure

“Retry three times” is not a reliability strategy.

Examples:

The workflow engine should understand enough about failures to choose the safe recovery path.

Use compensation when actions cannot simply be rolled back

Distributed workflows often perform multiple irreversible or externally visible actions.

Example:

reserve inventory
   ↓
charge payment
   ↓
create shipment

If shipment creation fails after payment succeeds, you cannot “rollback the transaction” across three external systems.

The workflow may need compensating actions:

shipment fails
   ↓
release inventory
   ↓
refund / void payment
   ↓
mark workflow compensated

Agentic systems still inherit normal distributed-systems problems.

LLMs do not make sagas, idempotency, transactional boundaries, or reconciliation disappear.

Design explicit stop conditions

A workflow needs to know what done means.

Weak design:

Keep researching until you have enough information.

Better:

Finish when:
- all four requested companies have been verified
- each material claim has at least one authoritative source
- unresolved conflicts are explicitly listed

Stop early if:
- a required source is inaccessible after two alternate attempts
- tool budget reaches 30 calls
- wall-clock budget reaches 8 minutes

A model is much easier to control when success and failure are both defined.

Do not use multi-agent systems by default

Multi-agent architecture can be excellent, but it is not a free upgrade over one agent.

Every extra agent creates:

• another context
• another prompt
• another source of nondeterminism
• more tool calls
• more latency or parallel compute
• more coordination logic
• more traces to debug

Anthropic's production research system found multi-agent designs particularly useful for breadth-first research where independent directions can be explored in parallel. It also found that they consume substantially more tokens and are a worse fit when subtasks depend heavily on one shared context.

So start with one agent unless you can explain why separation helps.

Good reasons include:

• independent parallel work
• genuinely different toolsets or permissions
• specialist prompts that significantly improve quality
• context isolation
• different models for different subtasks

“Multi-agent sounds sophisticated” is not a reason.

Two useful multi-agent orchestration styles

OpenAI's Agents SDK describes two especially useful patterns.

Manager / agents-as-tools

One central agent remains responsible for the user-facing task and calls specialists as tools.

User
  ↓
Manager
  ├→ research specialist
  ├→ database specialist
  └→ policy specialist
  ↓
Manager synthesizes
  ↓
User

Use this when:

• one component should own the final answer
• specialists perform bounded subtasks
• outputs need synthesis
• global policy should remain centralized

Handoffs

A routing agent transfers control to a specialist.

User
  ↓
Triage
  ↓
Refund specialist
  ↓
User

Use this when:

• one specialist should become the active expert
• the specialist should directly continue the conversation
• the new agent needs its own focused instructions/tools

Do not choose between them based on aesthetics. Choose based on who should own the workflow after delegation.

Context should be designed per step

A common workflow mistake is carrying the entire accumulated history into every model call.

Longer context is not automatically better context.

Anthropic's current context-engineering guidance treats context as a finite attention budget and recommends finding the smallest high-signal set of tokens that maximizes the chance of success.

For each step ask:

What does this model call actually need?

A billing classifier may need:

user message
invoice summary
allowed categories

It probably does not need:

all previous tool logs
full policy handbook
unrelated user memories
20 unused tool schemas

This lowers cost and reduces confusion.

Long-running workflows need compaction or durable notes

A long agent run can accumulate enormous context:

search results
file contents
logs
failed attempts
retries
subagent reports
tool outputs

Do not preserve all of it forever.

Compact completed phases into durable high-signal state:

Completed:
- identified root cause in retry handler
- confirmed duplicated POST after ambiguous timeout
- unit test reproduces issue

Decision:
- use stable idempotency key per logical payment operation

Remaining:
- patch handler
- run integration test

For long-horizon tasks, current agent engineering guidance increasingly uses a mix of compaction, structured notes, and specialized subagents with clean contexts.

Tool design is part of workflow design

The workflow can only be as reliable as its tools.

Bad tool surface:

execute_any_sql(query)
call_any_url(url, body)
run_shell(command)

Sometimes those capabilities are justified—especially in sandboxed coding environments—but they drastically widen the action space.

Business workflows usually benefit from narrower tools:

get_invoice(invoice_id)
search_customer_orders(customer_id, cursor)
propose_refund(invoice_id, amount, reason)
execute_approved_refund(proposal_id, idempotency_key)

Clear tools reduce ambiguity and make authorization, auditing, and testing much easier.

Guardrails belong at multiple boundaries

A single system prompt that says “be safe” is not a guardrail architecture.

Useful checks can exist at:

input
routing
retrieval
before tool call
after tool call
before mutation
before handoff
final output

OpenAI's Agents SDK, for example, separates input/output guardrails from tool-level guardrails because they run at different points in a workflow.

High-impact checks should be blocking and deterministic where possible.

Examples:

• authentication
• authorization
• tenant isolation
• payment limits
• destructive-action approval
• schema validation
• file/path boundaries
• network allowlists

These should not depend solely on model compliance.

Every workflow needs budgets

Autonomy without a budget is an outage waiting to happen.

Set limits such as:

max_iterations
max_model_calls
max_tool_calls
max_parallel_workers
max_subagents
max_writes
max_tokens
max_cost
max_wall_clock_time

The exact values depend on the product.

A support workflow may need seconds.

A repository migration agent may legitimately run for hours.

The architectural requirement is the same: the outer runtime—not the model—owns resource limits.

Trace the trajectory, not only the final answer

A final answer can look correct even when the workflow behaved badly.

Suppose an agent eventually returns the right result after:

14 unnecessary tool calls
2 unauthorized attempts
3 duplicate searches
1 failed mutation retry

A final-output-only metric may call that a success.

Production tracing should record the logical trajectory:

run
 ├─ route: billing_dispute
 ├─ tool: get_invoice
 ├─ retrieval: refund_policy
 ├─ model: propose_resolution
 ├─ guard: refund_limit
 ├─ approval: requested
 ├─ approval: granted
 ├─ tool: create_refund
 ├─ verification: refund_exists
 └─ final_response

OpenAI's Agents SDK traces generations, tool calls, handoffs, guardrails, and custom events. Google’s current agent tooling similarly evaluates tool selection, multi-turn trajectory quality, error recovery, and end-to-end task success—not just prose quality.

That is the right mental model for your own observability too.

Evaluate workflow behavior at several levels

Step-level evals

Did a router choose the correct branch?

Did extraction produce the right schema?

Did an evaluator catch the intended defect?

Tool-use evals

Did the agent choose the right tool?

Were arguments correct?

Did it avoid duplicate calls?

Did it respond correctly to tool errors?

Trajectory evals

Was the sequence of steps efficient and logically valid?

Did it recover from failure?

Did it stop when the goal was complete?

End-to-end task evals

Did the user actually get the intended result?

Was it safe?

Was it within latency and cost limits?

The final answer is only one artifact of the workflow.

Example: design a production incident-investigation workflow

Suppose a user asks:

> “Why did checkout latency spike after yesterday's deployment?”

A weak implementation is one giant agent with access to every infrastructure tool.

A better workflow can mix deterministic structure with bounded exploration.

Stage 1 — Intake

Application code records:

{
  "service": "checkout",
  "time_window": "since yesterday deployment",
  "goal": "identify likely cause with evidence"
}

Stage 2 — Deterministic baseline collection

Run known cheap checks in parallel:

recent deployments
latency/error metrics
dependency health
resource saturation

These do not need model reasoning to start.

Stage 3 — Agentic investigation

Give the model the baseline and a bounded set of diagnostic tools.

The agent may decide:

latency increased only on payment requests
→ inspect payment dependency
→ compare traces before/after deploy
→ inspect code diff touching payment client

This is where autonomy is valuable because the next step depends on what the system discovers.

Stage 4 — Parallel specialists if justified

If the incident is broad, the orchestrator might launch:

worker A → code-change analysis
worker B → trace/dependency analysis
worker C → historical incident search

Each receives a narrow scope and returns structured findings.

Stage 5 — Evidence gate

Before claiming root cause, require:

at least two independent supporting signals
or
one direct causal reproduction

If evidence is insufficient, the workflow should say so instead of inventing certainty.

Stage 6 — Proposed remediation

The agent can generate a fix plan.

It should not automatically modify production just because it found a plausible cause.

Stage 7 — Human checkpoint

Show:

root-cause hypothesis
evidence
proposed change
risk
rollback plan

Then require approval before a production mutation.

Stage 8 — Verify

After remediation, automatically check whether:

latency recovered
error rate normalized
no new regression appeared

Now the workflow has an actual closed loop rather than ending at “agent says fixed.”

How to choose the right pattern

Do not choose the most autonomous pattern.

Choose the least autonomous pattern that solves the task reliably.

A practical build order

1. Define success before writing prompts

Write the acceptance condition in plain language.

2. Build the deterministic skeleton

Encode known business steps and invariants in code.

3. Insert LLM decisions only where ambiguity exists

Start with the smallest possible agentic surface.

4. Give each LLM step a schema

Make stage boundaries machine-checkable.

5. Add tools gradually

Start read-only. Introduce writes with authorization, idempotency, and approvals.

6. Add durable state

Make important runs resumable and inspectable.

7. Add tracing before adding autonomy

You cannot improve behavior you cannot see.

8. Build eval cases from real failures

Evaluate routing, tool use, trajectories, and task completion.

9. Add loops only with stop conditions

Every feedback loop needs a cap.

10. Add multi-agent patterns only when one agent becomes the bottleneck

Complexity should be earned by evidence.

Common mistakes

Making every workflow autonomous

If the steps are known, use code.

Hiding business rules inside prompts

Critical rules should be enforceable without trusting model obedience.

No explicit state machine

If nobody can tell whether the run is planning, waiting for approval, executing, compensating, or complete, debugging becomes painful.

Passing prose between every step

Prefer structured contracts where the next component expects specific fields.

Unbounded reflection

“Keep improving until perfect” is an infinite loop disguised as a prompt.

Retrying writes like reads

A timeout after a side effect may mean success, not failure.

Too many overlapping tools

Tool ambiguity becomes decision ambiguity.

Premature multi-agent design

Five agents do not automatically outperform one good agent with five well-designed tools.

Evaluating only final text

Trajectory quality and action correctness matter more for agents than stylistic output scores.

Treating the framework as architecture

Your workflow semantics should survive a future change of model or orchestration library.

Anthropic's 2026 Managed Agents engineering work makes a related point: harness assumptions can become stale as models improve, so stable interfaces and responsibilities age better than layers of model-specific tricks.

Production checklist

Before shipping an agentic workflow, make sure you can answer:

• What is the exact success condition?
• Which steps are deterministic and which are model-driven?
• Why does every agentic decision need an LLM?
• What state is durable across retries/restarts?
• Are inputs and outputs structured at important boundaries?
• Which tools are read-only versus mutating?
• Are permissions enforced outside the model?
• Are mutations idempotent?
• Can ambiguous write outcomes be verified?
• What are the retry rules for each failure class?
• Where are human approvals required?
• What are the maximum step/tool/token/time/cost budgets?
• What stops every loop?
• Can parallel workers duplicate or conflict with each other?
• Can the workflow resume after a crash?
• Are compensation/reconciliation paths defined where needed?
• Can you inspect the entire trajectory in traces?
• Do evals test tool selection and trajectories, not only final answers?
• Can a new model be adopted without redesigning the whole workflow?

If several answers are “the model handles it,” the design probably has too little structure.

Final takeaway

Agentic workflow design is not about maximizing autonomy.

It is about placing autonomy exactly where it creates value.

Use deterministic software for:

authentication
authorization
business invariants
known control flow
state transitions
budgets
idempotency
retries
approvals

Use models for:

language understanding
semantic routing
open-ended decomposition
planning under ambiguity
synthesis
exploration
judgment

Then compose the two with explicit state, clear contracts, bounded loops, good tools, tracing, and evaluation.

A useful production principle is:

> If you already know the next step, let code choose it. If the next step depends on understanding an ambiguous situation, let the model help choose it—inside a boundary the application still controls.

That is what makes an agentic workflow flexible without making it uncontrollable.

References and further reading

• Anthropic — Building Effective Agents
• Anthropic — Effective Context Engineering for AI Agents
• Anthropic — How We Built Our Multi-Agent Research System
• Anthropic — Scaling Managed Agents: Decoupling the Brain from the Hands
• OpenAI — A Practical Guide to Building AI Agents
• OpenAI Agents SDK — Agent Orchestration
• OpenAI Agents SDK — Guardrails
• OpenAI Agents SDK — Tracing
• OpenAI Agents SDK — Testing
• Google Agents CLI — Evaluation Guide