The AI Coding Agent War Is No Longer About Models — It’s About the Harness

`The model may write the code, but the harness decides what it sees, what it can do, how safely it acts, and whether the result is actually ready to ship.

Give two AI coding products the same model, the same repository, and the same task. One agent finds the correct service, traces the execution path, edits the relevant files, runs the right tests, detects an edge case, and returns a clean, reviewable change. The other searches randomly, reads half the repository, edits the wrong abstraction, consumes its context window, and confidently announces:

“The issue has been fixed.”

The model may be identical. The difference is the harness. For the first generation of AI coding tools, the competition was largely about model intelligence: stronger code completion, better reasoning, larger context windows, and higher benchmark scores. Those things still matter. But they are no longer the whole battle. The new competition is about everything surrounding the model:

• How the agent gathers and prioritizes context
• How it searches and understands a repository
• Which tools it can use
• How those tools are described and secured
• Where commands are executed
• Which actions require approval
• How failures are represented
• How context is compacted during long tasks
• How work survives across sessions
• How code changes are tested and verified
• How developers inspect and control the run

That surrounding execution system is the agent harness. OpenAI defines the harness as the system around the model that maintains context, uses tools, exposes progress, handles failure, requests approval, and returns useful results. Its Codex harness manages conversation state, streaming execution, tools, sandbox policies, approvals, and multi-turn work. DeepSeek expresses the separation in an even simpler phrase: “Agent = Model + Harness.” OpenAI Codex platform documentation DeepSeek Harness

The model supplies intelligence. The harness turns that intelligence into reliable action.

Table of Contents

1. The Model Is No Longer the Whole Agent
2. The Evidence: Same Model, Different Harness, Different Result
3. What Is an Agent Harness?
4. Context Engineering
5. Tool Design
6. The Agent Loop
7. Safe Code Editing
8. Sandboxing and Permissions
9. Verification and Recovery
10. Memory and Sessions
11. Observability
12. Multi-Agent Orchestration
13. The Product Experience
14. How Leading Platforms Reveal the Strategy
15. Why Models Still Matter
16. A Realistic Example
17. Production Harness Blueprint
18. How to Evaluate Coding Agents
19. What Comes Next
20. Final Thought
21. FAQ
22. References

The Model Is No Longer the Whole Agent

A language model can reason about code. It can propose an implementation, generate a function, explain an error, or recommend an architectural change. But a model by itself cannot reliably:

• Discover which files matter in an unfamiliar repository
• Determine which documentation is current
• Decide how much context should be loaded
• Safely execute shell commands
• Prevent an edit from overwriting concurrent work
• Recover from a failed tool invocation
• Track progress across a long-running task
• Enforce filesystem, network, and credential boundaries
• Prove that the final implementation satisfies the request

Those responsibilities belong to the harness. A useful mental model is:

Agent Performance ≈ Model Capability
  × Context Quality
  × Tool Quality
  × Execution Reliability
  × Verification Strength
  × Safety and Recovery

This is not a literal scientific formula. It is an engineering model. The important point is that agent quality behaves more like a multiplicative system than an additive one. A brilliant model with poor context can solve the wrong problem. A brilliant model with unreliable tools can corrupt files or repeat failed actions. A brilliant model without verification can generate plausible but incorrect code. A brilliant model with unrestricted system access can become a security incident.

The model determines the potential ceiling. The harness determines how much of that potential becomes dependable engineering work.

The Evidence: Same Model, Different Harness, Different Result

The importance of the harness is no longer only an architectural opinion. Researchers are beginning to measure it directly.

Important research note: Several 2026 results discussed below are recent preprints. They are useful evidence, but they should not be treated as the final word on agent evaluation.

1. Same Model, Different Harness

A 2026 preprint kept model weights and tasks fixed while changing how the harness handled old tool results, context pressure, and repeated or stalled work. Under a constrained 20,480-token window on a 169-task SWE-bench Verified cohort, the modified harness increased complete solutions from 43 to 72.

The same frozen harness changes also improved outcomes for additional model families without model-specific tuning. Same Model, Different Harness The lesson is not that one compaction strategy solves every agent problem. The lesson is that an unchanged model can produce materially different results when the surrounding execution system changes.

2. The Scaffold Effect

Another 2026 preprint evaluated the same models across Goose, OpenCode, and OpenHands-SDK on a subset of Terminal-Bench Pro. Harness choice produced up to a 40× difference in tokens consumed per solved task.

Pass-rate differences in that experiment were smaller—generally between zero and eight percentage points—but each harness exhibited distinct failure patterns, including excessive reasoning, verification failures, idle loops, and maximum-turn exhaustion. The Scaffold Effect in Coding Agents That distinction is commercially important. Two harnesses can achieve similar pass rates while having radically different:

• Cost
• Latency
• Human oversight requirements
• Failure behaviour
• Operational predictability

3. SWE-agent and the Agent-Computer Interface

The 2024 SWE-agent work demonstrated that interface design can substantially affect how language-model agents navigate repositories, edit code, and run tests. Its central argument was that agents benefit from purpose-built computer interfaces rather than being dropped into an unrestricted shell with weak feedback. SWE-agent paper This established an enduring principle:

How an agent interacts with a computer can be almost as important as what the model knows.

What These Studies Do—and Do Not—Prove

These studies do not prove that the harness always matters more than the model. Model capability remains fundamental, and results depend on the task, benchmark, context limit, toolset, time budget, and evaluation method. They prove something more practical:

A model name alone is no longer enough to describe the system being evaluated.

The real comparison unit is:

Model + Harness + Tools + Context Policy + Execution Environment + Verification Policy

When teams compare “Model A versus Model B” through different products, they may actually be comparing two complete systems—not two models.

What Is an Agent Harness?

The word harness is used in slightly different ways across the industry. In the broad product sense, it is the complete execution layer surrounding the model. A production coding-agent harness typically owns:

1. Prompt and context construction
2. Repository discovery and code search
3. The reasoning and tool-execution loop
4. File editing and command execution
5. Context compaction and session persistence
6. Sandboxing and permission enforcement
7. Testing and verification
8. Error recovery and loop detection
9. Tracing, observability, and replay
10. Human approval and intervention
11. Skills, plugins, hooks, and MCP integrations
12. Subagent and parallel-work orchestration

A high-level architecture looks like this:

The harness is not just middleware forwarding messages between a user and a model. It is a stateful software runtime.

1. Context Engineering: Showing the Model the Right Reality

Prompt engineering asks:

What instructions should we give the model?

Context engineering asks:

What information should the model see at this exact step—and what should be excluded?

That second question becomes crucial in large repositories. A coding agent may have access to:

• Thousands of source files
• Years of Git history
• Architecture documents
• Issue descriptions
• Build logs
• Test failures
• Database schemas
• API specifications
• Previous agent turns
• User corrections
• External documentation

Dumping all of this into the model is not a strategy. It creates noise, increases cost, and can hide the most important constraint inside irrelevant material. A strong context engine must decide:

• Which repository instructions apply to the current path
• Which services and modules are relevant
• Which files should be read fully
• Which files require only selected ranges
• Which symbols and references should be explored
• Which previous decisions remain valid
• Which tool outputs should be summarized
• Which logs can be discarded
• What must remain pinned during compaction
• When a subagent should isolate a noisy investigation

Larger Context Is Not the Same as Better Context

A huge context window can help, but it does not remove the need for selection. A million tokens of stale documentation, generated files, duplicate logs, and irrelevant code can perform worse than a much smaller context containing:

• The user’s real objective
• Applicable repository instructions
• The relevant call chain
• The current schema
• The failing tests
• The constraints that define success

A mature context manager should provide:

• Repository-aware search
• Symbol and dependency discovery
• Hierarchical project instructions
• Selective document retrieval
• Tool-output summarization
• Context deduplication
• Token budgeting
• Long-session compaction
• Explicit preservation of goals, decisions, and unfinished work

OpenAI’s documentation on long-running agents describes compaction as a way to keep stateful work moving when workflows approach context limits. OpenAI: Shell, Skills, and Compaction

Progressive Disclosure Beats Permanent Prompt Inflation

Not every skill, policy, tool description, and project document should be loaded at startup. A better pattern is progressive disclosure:

1. Start with a compact system contract.
2. Discover relevant repository instructions.
3. Load a skill only when the task needs it.
4. Retrieve files only when evidence points toward them.
5. Summarize completed investigations before moving on.

The winning harness will not be the one that reads the most code. It will be the one that consistently reads the right code.

2. Tool Design: Giving the Agent Better Hands

A model cannot inspect a repository, edit a file, run tests, or query an external service through intention alone. It needs tools. Typical coding-agent tools include:

• Search files and symbols
• Read files or selected ranges
• Find references
• Apply targeted edits
• Create, move, or delete files
• Execute shell commands
• Inspect Git status and diffs
• Run tests, builds, linters, and type checks
• Search documentation
• Interact with issue trackers
• Query databases and observability systems
• Invoke external services through MCP

But more tools do not automatically create a better agent.

Tool Overload Is a Real Failure Mode

If an agent is exposed to ten overlapping search tools, several generic shell tools, and dozens of poorly named MCP operations, it must first solve a tool-selection puzzle before solving the user’s problem. Weak tool design causes:

• Incorrect tool selection
• Invalid parameters
• Excessive tool descriptions in context
• Huge unstructured outputs
• Repeated permanent failures
• Ambiguous completion states
• Unsafe retries
• Permission confusion

A strong tool contract should include:

name: run_targeted_tests
purpose: Run a bounded test selection for the current change.
inputs:
  test_paths:
    type: array
    items: string
  timeout_seconds:
    type: integer
    minimum: 1
    maximum: 900
permissions:
  filesystem: read
  network: denied
  workspace_write: temporary-test-output-only
behaviour:
  cancellable: true
  retry_safe: true
  max_output_chars: 30000
outputs:
  status: passed | failed | timed_out | cancelled | environment_error
  discovered_tests: integer
  passed: integer
  failed: integer
  summary: string
  bounded_log: string

The exact schema will differ across systems, but the principles are stable:

• One clear responsibility
• Typed inputs
• Structured outputs
• Bounded response size
• Explicit error categories
• Cancellation and timeout support
• Safe retry semantics
• Permission metadata
• Auditability

Tool Errors Should Teach the Agent What to Do Next

Bad error:

Command failed.

Useful error:

{
  "status": "permission_denied",
  "reason": "Network access is disabled in workspace-write mode",
  "recoverable": true,
  "allowed_next_actions": [
    "request_network_approval",
    "use_cached_dependencies",
    "continue_without_installation"
  ]
}

The purpose of an error is not only to report failure. It should help the orchestrator choose the correct recovery path.

MCP Is an Integration Protocol, Not the Entire Harness

The Model Context Protocol allows servers to expose discoverable tools with schemas. The current specification recommends visible tool activity and keeping a human able to deny invocations. MCP tools specification MCP can be an important tool layer, but it does not automatically provide:

• Agent planning
• Context management
• Sandbox isolation
• Retry logic
• Session persistence
• Verification
• Cost control
• Observability
• Approval UX

“Supports MCP” is therefore not the same as “has a production-ready harness.”

3. The Agent Loop: Turning a Response Into a Workflow

A chatbot follows a simple interaction:

Prompt → Response

A coding agent needs a loop:

Observe → Understand → Plan → Act → Inspect → Verify → Continue or Stop

The harness manages this loop. It sends the task and context to the model. The model requests a tool. The harness validates permissions, executes the tool, records the result, updates the task state, and asks the model what to do next. A simplified version might look like this:

while run.status is active:
    context = context_manager.build(run)
    decision = model.next_action(context)

    if decision.type == "tool_call":
        policy_result = policy_engine.evaluate(decision)
        if policy_result == DENY:
            run.add_event(tool_denied(decision))
            continue
        if policy_result == ASK:
            run.status = WAITING_FOR_APPROVAL
            surface_approval(decision)
            continue
        result = tool_gateway.execute(decision)
        run.add_event(result)
        recovery_engine.update(result)
        continue

    if decision.type == "complete":
        evidence = verifier.check(run)
        if evidence.sufficient:
            run.status = COMPLETED
        else:
            run.add_event(verification_failed(evidence))
            run.status = RECOVERING

The hard part is not the loop itself. The hard part is controlling failure. A production loop needs answers to questions such as:

• How many retries are allowed?
• Which failures are temporary?
• Which failures require a new plan?
• How do we detect repeated actions?
• What counts as meaningful progress?
• When should the agent stop searching?
• When should it request clarification?
• How can the user interrupt or steer the run?
• What happens after context compaction?
• What evidence is required before completion?

Failure Classification Matters

These failures should not be handled identically:

A weak loop forwards every failure back to the model as plain text. A strong loop classifies failures, controls retries, protects state, and makes the next action more informed.

4. Safe Code Editing Is a Systems Problem

Generating a correct code snippet is easier than modifying a living repository safely. A production editing layer must handle:

• Multiple edits in the same file
• Files changing between read and write
• Line-ending differences
• Formatting tools rewriting content
• Generated files
• Binary files
• Symlinks and path traversal
• Monorepo boundaries
• Partially applied patches
• Concurrent agents modifying related files
• Rollback after a failed implementation
• Very large files
• Restricted directories

Editing Should Be Optimistic but Conflict-Aware

A safe editing tool can require a content hash or version from the original read:

{
  "path": "src/payments/invoice-service.ts",
  "expected_sha256": "9f87...",
  "edits": [
    {
      "start_line": 118,
      "end_line": 132,
      "replacement": "..."
    }
  ]
}

If the file changed after the model read it, the tool should reject the edit rather than silently overwriting new work.

Changes Should Be Atomic Where Possible

For multi-file work, a harness can:

1. Create a checkpoint.
2. Validate all proposed paths.
3. Apply edits to a temporary workspace.
4. Run formatting and focused checks.
5. Commit the complete edit set only after validation.
6. Restore the checkpoint if the operation fails irrecoverably.

Gemini CLI documents automatic checkpoints before AI-powered file modifications so users can restore earlier project state. Claude’s Agent SDK exposes file checkpointing and rewind features as part of its control layer. Gemini CLI checkpointing Claude Agent SDK A coding agent should not merely know how to write code. Its harness must know how to change a repository without damaging it.

5. Sandboxing and Permissions: Autonomy Without Recklessness

Coding agents are useful because they can act. That is also what makes them dangerous. An agent with shell and network access may accidentally:

• Delete files outside the repository
• Read credentials
• Send secrets to an external service
• Execute malicious project scripts
• Modify system configuration
• Install unsafe dependencies
• Trigger an irreversible database operation
• Use an untrusted MCP tool

The correct response is not to eliminate autonomy. The correct response is to define the execution boundary precisely.

Separate the Control Plane From the Execution Plane

OpenAI’s sandbox-agent documentation describes a useful split:

• The harness is the control plane: model calls, orchestration, approvals, tracing, recovery, and run state.
• The sandbox is the execution plane: files, commands, packages, ports, mounted storage, and snapshots.

Keeping those boundaries separate allows trusted infrastructure to retain authentication, billing, audit logs, and approval state while model-directed code runs inside a narrower environment. OpenAI sandbox-agent documentation

Permissions Should Be Capability-Based

A mature system should distinguish between:

• Read-only repository access
• Workspace write access
• External directory access
• Network access
• Package installation
• Secret access
• Destructive commands
• External service writes
• Production operations

For example:

Read repository file → Allow
Search Git history → Allow
Run existing unit tests → Allow
Write inside workspace → Allow in workspace-write mode
Install a new package → Ask
Access an external domain → Ask or allowlist
Read production credentials → Deny
Drop a production database → Deny
Deploy to production → Separate privileged workflow

Gemini CLI’s policy engine supports rules that allow, deny, or require confirmation for tool calls. Its sandbox expansion flow can request additional access for a specific command when restricted paths or network access are needed. Gemini CLI policy engine Gemini CLI sandboxing OpenAI’s Codex guidance similarly separates sandbox mode from approval mode and recommends starting with tight permissions. Codex sandbox best practices

Prompts Are Not Security Boundaries

“Never read secrets” is useful guidance. It is not sufficient enforcement. Real controls should exist below the model:

• Filesystem mounts
• Credential brokers
• Network proxies
• Domain allowlists
• Command policies
• Resource quotas
• Approval gates
• Secret redaction
• Process cleanup

A useful design principle is:

Increase autonomy inside a trustworthy boundary instead of removing the boundary.

6. Verification Is What Separates Generation From Engineering

Many weak coding agents use this completion rule:

Edit applied → Task complete

A production agent should use something closer to:

Edit applied
↓
Inspect diff
↓
Run focused tests
↓
Run type check, lint, or build
↓
Analyze failures
↓
Repair or re-plan
↓
Run broader validation when justified
↓
Confirm requirements
↓
Task complete

Successful tool execution is not proof of a successful implementation. A test command may exit successfully while discovering zero tests. A build may pass while the requested behavior remains missing. A unit test may pass while another service breaks. A migration may compile while being unsafe for a large production table.

Verification Should Check the Claim, Not Just the Command

A strong verifier should ask:

• Does the requested behavior now exist?
• Were relevant tests discovered and executed?
• Did existing behavior regress?
• Does the diff contain unrelated changes?
• Are generated files current?
• Are migrations safe and reversible?
• Were debug statements or secrets introduced?
• Does the implementation follow architectural constraints?
• Does the evidence justify the completion claim?

Verification Should Feed the Next Iteration

The strongest pattern is a closed loop:

Implementation
↓
Verification failure
↓
Structured diagnosis
↓
Updated context
↓
Targeted correction
↓
Re-verification

A useful verification result might look like:

{
  "status": "failed",
  "checks": [
    {
      "name": "targeted_tests",
      "status": "failed",
      "discovered": 18,
      "passed": 17,
      "failed": 1,
      "failure_summary": "Duplicate invoice created under concurrent retry"
    },
    {
      "name": "typecheck",
      "status": "passed"
    }
  ],
  "recommended_next_step": "Inspect transaction boundary and idempotency constraint"
}

OpenAI’s iterative-repair guidance describes agent workflows that generate an output, validate it, and use feedback for the next pass. OpenAI iterative repair loops The model can claim the task is complete. The harness must demand evidence.

7. Memory, Sessions, Checkpoints, and Recovery

Real engineering tasks do not always finish in one model call. An agent may need to:

• Continue for hours
• Pause for human approval
• Resume after an application restart
• Recover from a failed command
• Revert an incorrect edit
• Fork an alternative implementation
• Preserve decisions across context compaction
• Continue after the user changes direction

This requires durable session state.

Memory Should Be Layered

A good system separates different kinds of memory.

Repository Instructions

Stable information such as:

• Architecture boundaries
• Build commands
• Testing conventions
• Security rules
• Definition of done

Task State

Information relevant to the active run:

• User goal
• Current plan
• Completed steps
• Files inspected
• Files changed
• Pending approvals
• Remaining work
• Verification status

Learned Preferences

Carefully selected recurring information:

• Review expectations
• Preferred workflows
• Common corrections
• Output format preferences

Ephemeral Tool Output

Information that can usually be summarized or discarded:

• Long logs
• Search result dumps
• Repeated directory listings
• Intermediate diagnostics

Memory Must Be Selective and Bounded

Remembering everything is not intelligence. Unbounded memory causes:

• Higher context cost
• Stale assumptions
• Repeated outdated decisions
• Privacy risks
• Difficulty identifying the real source of truth

The right question is not:

How much can the agent remember?

It is:

What must the agent remember to continue correctly?

Sessions Should Be Event-Based

Instead of storing only a transcript, a harness can persist structured events:

{
  "event_id": "evt_01842",
  "run_id": "run_771",
  "timestamp": "2026-08-29T14:26:18Z",
  "type": "tool_result",
  "tool": "run_targeted_tests",
  "status": "failed",
  "artifacts": [
    "test-report.json"
  ],
  "context_policy": {
    "retain_summary": true,
    "retain_full_output": false
  }
}

Event-based state enables:

• Resume
• Replay
• Forking
• Auditing
• Context reconstruction
• Crash recovery
• Cost attribution

DeepSeek Harness records model-visible prompts, reasoning, tool calls, results, subagent scheduling, and context injections in an append-only session log. Resume, fork, search, and replay operate over the same event stream. DeepSeek Harness Anthropic’s Agent SDK exposes sessions, resumption, permissions, hooks, subagents, checkpointing, usage tracking, and OpenTelemetry support. Claude Agent SDK Long-running agents need more than chat history. They need durable execution state.

8. Observability Is the Difference Between a Demo and a Production System

When a normal application fails, engineers inspect logs, metrics, traces, and database state. AI agents need the same operational discipline, plus agent-specific visibility. A production trace should help answer:

• What instructions were active?
• Which context sources were selected?
• Which files did the agent inspect?
• Which tools did it call?
• What arguments were passed?
• How long did each step take?
• Which permission decisions occurred?
• Where did retries happen?
• When was context compacted?
• Did a subagent contribute?
• What did the run cost?
• Why did the agent stop?
• What evidence supported completion?

A useful trace hierarchy might look like:

Run
├── User request
├── Instruction resolution
├── Context retrieval
├── Model call
├── Tool request
├── Permission decision
├── Tool execution
├── Tool result
├── Context compaction
├── Model continuation
├── File changes
├── Verification
└── Final response

Metrics That Actually Matter

Useful production metrics include:

“Number of messages generated” is not a success metric. “Useful changes accepted without regression” is.

Traces Should Create an Improvement Flywheel

Every production failure can become:

• A new evaluation case
• A better tool schema
• A stronger policy
• A context-selection improvement
• A new recovery rule
• A more precise verifier
• A better skill

OpenAI’s agent-improvement guidance explicitly connects traces, feedback, evaluations, and harness changes into an improvement loop. OpenAI agent improvement loop Without traces, teams repeatedly fix symptoms. With traces, they can improve the system that produced the failure.

9. Multi-Agent Orchestration: Useful Tool or Expensive Theatre?

Multi-agent systems sound impressive:

• One agent plans
• One investigates the backend
• One handles the frontend
• One reviews security
• One writes tests
• One supervises the others

Sometimes this works extremely well. Sometimes it is five agents repeatedly summarizing the same repository while multiplying cost and coordination failure.

Use Subagents When Work Is Genuinely Separable

Good candidates include:

• Independent architecture research
• Security review
• Test analysis
• Documentation validation
• Parallel investigation of unrelated failures
• Frontend and backend work behind a stable contract

Poor candidates include:

• Tightly coupled edits to the same files
• Tasks with unclear ownership
• Work requiring continuous shared state
• Small changes where delegation costs more than execution

Each subagent should receive:

• A narrow objective
• Minimal relevant context
• Explicit tools
• Scoped permissions
• A bounded time and token budget
• A structured output contract
• Clear stop conditions

The parent should receive the result—not the child’s entire working history. Anthropic and Gemini both expose specialized subagents as part of their agent runtimes, while DeepSeek Harness includes subagents and workflows in its standard mode. Claude Agent SDK Gemini CLI subagents DeepSeek Harness The future is probably not the largest possible swarm. It is small, specialized, observable delegation.

10. The Product Experience Is Part of the Harness

Developers do not experience a model directly. They experience:

• How quickly meaningful progress appears
• Whether tool activity is understandable
• Whether the agent can be interrupted
• Whether new instructions can steer the active run
• How approvals are presented
• How diffs are reviewed
• Whether failed work can be resumed
• Whether changes can be reverted
• Whether uncertainty is communicated honestly

A strong interface should distinguish between states such as:

Understanding
Searching
Reading
Planning
Editing
Running tests
Waiting for approval
Recovering
Blocked
Completed

It should show enough information to support trust without overwhelming users with every internal detail.

Chat Is Not the Only Agent Interface

A reusable harness can power:

• IDE assistants
• Issue-tracker agents
• Pull-request reviewers
• Operations dashboards
• Database migration assistants
• Security remediation systems
• Internal developer portals
• Domain-specific engineering workflows

OpenAI’s app-server direction separates the reusable Codex agent loop from the application-owned interface, business context, tools, and approval flow. OpenAI Codex platform documentation The interface is not decoration. It is where context, consent, progress, and evidence become visible to humans.

How Leading Platforms Reveal the New Strategy

The industry’s major coding-agent platforms increasingly expose harness capabilities as products of their own.

Sources: OpenAI Codex platform documentation, Claude Agent SDK, Gemini CLI policy engine, Gemini CLI checkpointing, and DeepSeek Harness. These products are not identical, and this table is not a quality ranking. The important observation is that all of them are investing heavily in capabilities that exist around inference. That is where product reliability is being built.

Models Still Matter—a Lot

The title of this article should not be misunderstood. Models remain fundamental. They determine capabilities such as:

• Understanding ambiguous requirements
• Navigating unfamiliar code
• Reasoning across multiple services
• Writing correct implementations
• Diagnosing failures
• Following instructions
• Selecting appropriate tools
• Maintaining coherence over long tasks
• Recognizing uncertainty
• Balancing quality, latency, and cost

A weak model cannot be transformed into a frontier engineer merely by surrounding it with excellent middleware. The accurate statement is:

The model sets the capability ceiling. The harness determines how consistently, safely, and efficiently the system reaches that ceiling.

Model and harness also interact. A tool interface that works well for one model may confuse another. A compaction strategy may preserve the cues one model needs while discarding information another relies on. Different providers may require different:

• Prompt structures
• Tool schemas
• Reasoning controls
• Streaming adapters
• Retry behavior
• Error handling
• Context policies

The future is not purely model competition or purely harness competition. It is model-harness co-design. But there is an important business difference. Model capability can often be rented through an API. A deeply integrated harness containing company-specific context, tools, policies, evaluations, workflows, and reliability improvements is much harder to copy. That is where a durable product advantage can emerge.

A Realistic Example: One Bug, Two Different Agents

Consider this task:

“Fix occasional duplicate invoices created when payment retries occur.”

A Weak Harness

The agent might:

1. Search for the word invoice.
2. Open the first matching controller.
3. Add an in-memory duplicate check.
4. Ignore queue retries and multiple server instances.
5. Never inspect database constraints.
6. Run one unrelated unit test.
7. Claim the bug is fixed.

The code may look reasonable. The system is still broken.

A Strong Harness

The same model inside a better harness might:

1. Resolve repository and service instructions.
2. Map payment, invoice, webhook, and queue components.
3. Inspect the database schema and uniqueness rules.
4. Find every invoice-creation call site.
5. Trace retry and concurrency behavior.
6. Identify whether the operation requires an idempotency key.
7. Produce a read-only implementation plan.
8. Create a workspace checkpoint.
9. Apply a transactional or database-level guard.
10. Add concurrent retry tests.
11. Run focused tests, type checks, and migration validation.
12. Inspect the final diff for unrelated changes.
13. Report evidence, residual risks, and rollback notes.

The model did not suddenly become smarter. The harness gave it:

• Better context
• Better tools
• Better constraints
• Better feedback
• Better verification

That is the thesis of this article in one example.

Why a Stronger Model Can Still Lose

This is why benchmark-leading models can feel disappointing inside poorly designed products. The product may not be exposing the model’s real capability.

A Production-Ready Coding Harness Blueprint

A serious coding-agent platform should keep its layers explicit.

1. Model Adapter Layer

Responsibilities:

• Support one or more providers
• Normalize tool-call formats
• Normalize streaming events
• Handle provider-specific reasoning modes
• Track usage and cost
• Support fallback and routing
• Handle cancellation
• Normalize transient and permanent errors
• Preserve provider-specific strengths instead of forcing a lowest-common-denominator interface

2. Instruction and Context Engine

Responsibilities:

• Resolve repository instructions by scope
• Search files, symbols, and dependencies
• Retrieve relevant documentation
• Deduplicate context
• Compress tool results
• Maintain token budgets
• Compact long sessions
• Preserve goals, decisions, risks, and pending work

3. Explicit Agent State Machine

Useful runtime states include:

UNDERSTANDING
PLANNING
EXECUTING
WAITING_FOR_APPROVAL
VERIFYING
RECOVERING
BLOCKED
COMPLETED
FAILED
CANCELLED

Avoid hiding all behavior inside one unobservable loop.

4. Tool Gateway

Every tool should define:

• Input schema
• Output schema
• Permission level
• Timeout
• Cancellation behavior
• Maximum output size
• Retry safety
• Error categories
• Audit metadata

The gateway should also provide:

• Tool discovery
• Health checks
• Output truncation with artifact storage
• Secret redaction
• Concurrency limits
• Per-run budgets

5. Policy and Approval Engine

Responsibilities:

• Allow, ask, or deny actions
• Apply filesystem boundaries
• Apply network policies
• Protect secrets
• Restrict external service writes
• Require stronger approval for destructive operations
• Prevent permission escalation
• Record every decision

6. Sandbox and Execution Layer

Responsibilities:

• Isolate commands
• Limit CPU, memory, processes, and time
• Control writable paths
• Restrict network access
• Mount only necessary files and credentials
• Clean up abandoned processes
• Support reproducible environments
• Capture output safely
• Snapshot and restore workspace state

7. Editing and Change Management

Responsibilities:

• Conflict-aware edits
• Atomic multi-file changes
• Checkpoints
• Rollback
• Diff generation
• Scope validation
• Path traversal protection
• Symlink policy
• Formatting integration

8. Verification Engine

Responsibilities:

• Select relevant tests
• Run builds, linters, and type checks
• Validate migrations
• Inspect diffs
• Enforce architecture rules
• Detect unrelated modifications
• Check completion claims against evidence
• Return structured failures to the agent

9. Session and Event Store

Responsibilities:

• Persist task state
• Store plans and progress
• Save tool calls and results
• Support pause, resume, fork, and replay
• Store checkpoints
• Preserve compaction summaries
• Provide crash recovery
• Track cost and latency

10. Observability and Evaluation

Responsibilities:

• Trace every run
• Track model and tool latency
• Record approvals and denials
• Measure task success
• Replay failures
• Compare models and harness versions
• Convert production incidents into evaluations
• Detect regressions before release

11. Product Experience

Responsibilities:

• Show meaningful progress
• Display tool use clearly
• Make approvals understandable
• Support interruption and steering
• Present diffs and validation evidence
• Separate planning from execution
• Make blockers honest and actionable

Reference Architecture

A Practical Event Model

A production harness should make every important transition inspectable.

type AgentEvent =
  | { type: "run.started"; runId: string; task: string }
  | { type: "context.selected"; sources: ContextSource[] }
  | { type: "model.started"; provider: string; model: string }
  | { type: "model.delta"; content: string }
  | { type: "tool.requested"; tool: string; arguments: unknown }
  | { type: "approval.requested"; risk: string; rationale: string }
  | { type: "approval.resolved"; decision: "allow" | "deny" }
  | { type: "tool.completed"; status: ToolStatus; resultRef: string }
  | { type: "files.changed"; paths: string[]; checkpointId: string }
  | { type: "context.compacted"; summaryRef: string }
  | { type: "verification.completed"; status: "passed" | "failed" }
  | { type: "run.blocked"; reason: string }
  | { type: "run.completed"; evidenceRefs: string[] }
  | { type: "run.failed"; error: StructuredError }
  | { type: "run.cancelled"; requestedBy: "user" | "policy" | "system" };

This model supports streaming UI, replay, auditing, recovery, and analytics without depending on fragile plain-text parsing.

How Teams Should Evaluate Coding Agents Now

Engineering teams should stop evaluating agents only through impressive chat demonstrations. Give each candidate the same real tasks and measure the complete workflow.

A Better Evaluation Scorecard

Do not ask only:

“Which model powers this agent?”

Also ask:

“What happens around the model before, during, and after every action?”

That second question usually reveals more about the product.

Measure Tokens Per Solved Task, Not Tokens Per Request

A cheap request that fails repeatedly is expensive. A powerful agent that consumes uncontrolled tokens can also be commercially unusable. The useful unit is not the cost of one model call. It is the cost of a verified, accepted result. Track:

Cost per accepted change =
  Total model cost
  + Tool infrastructure cost
  + Human review cost
  + Recovery cost
  + Regression cost

This is why the Scaffold Effect’s focus on tokens per solved task is valuable. Similar pass rates can hide enormous differences in efficiency and human waiting time. The Scaffold Effect in Coding Agents

Evaluate Models and Harnesses Separately

Use a two-dimensional matrix:

When publishing or reviewing results, disclose:

• Exact model and version
• Reasoning configuration
• System and developer instructions
• Available tools
• Editing interface
• Context-selection strategy
• Context limit and compaction policy
• Sandbox permissions
• Network access
• Maximum runtime and turns
• Verification steps
• Retry rules
• Human intervention
• Token and cost accounting

Without these details, an agent benchmark is not fully reproducible. It is a score for an underspecified composite system.

Build an Internal Evaluation Suite

Public benchmarks are useful, but production teams need tasks drawn from their own environment. A strong internal suite should include:

• Real bugs from the company’s repositories
• Multi-file feature work
• Database migrations
• Security-sensitive changes
• Large-log debugging
• Frontend and backend integration
• Hidden edge cases
• Tasks designed to expose repeated loops
• Tasks that require stopping instead of editing
• Tasks where the correct response is to ask for approval
• Tasks where a partial result is better than an unsafe guess

Every escaped production failure should become a regression case.

Build, Buy, or Extend?

Most companies should not rebuild every low-level agent component from zero. A practical strategy is:

1. Start with an existing runtime or SDK for the basic loop.
2. Keep provider adapters modular.
3. Build company-specific context and tools.
4. Enforce business-specific security policies.
5. Add verification tied to the real definition of done.
6. Trace every run.
7. Convert failures into evaluations. Invest custom engineering where it creates real differentiation:

• Domain tools
• Repository understanding
• Security controls
• Workflow integrations
• Verification pipelines
• Operational reliability
• User experience

Do not build a custom orchestrator merely because agent loops look exciting. Build one when existing runtimes cannot satisfy a concrete product or reliability requirement.

Where the Coding-Agent War Goes Next

1. Model Switching Will Become Normal

Teams will increasingly route tasks based on:

• Capability
• Cost
• Latency
• Context size
• Tool reliability
• Data restrictions
• Specialized strengths

The harness will preserve the product experience while the underlying model changes.

2. Harness Benchmarks Will Become a Separate Category

Evaluation will increasingly distinguish between:

• Raw model capability
• Standardized-harness performance
• Full product-agent performance

This will make comparisons more honest.

3. Vertical Harnesses Will Beat Generic Chat Wrappers

A harness designed for database migrations, security remediation, incident response, mobile development, or frontend delivery can provide better context and safer tools than a universal chat interface. The winning product may not be the one that can theoretically do everything. It may be the one that reliably completes one valuable category of work.

4. Agent Observability Will Become an Engineering Discipline

Teams will need dashboards for:

• Agent traces
• Failure categories
• Tool reliability
• Token economics
• Context quality
• Permission behavior
• Verification success
• Security events

Agent operations will become as important as application operations.

5. Security Will Become a Buying Criterion

Enterprises will compare agents based on:

• Data boundaries
• Sandboxing
• Approval controls
• Secret handling
• MCP governance
• Auditability
• Policy enforcement
• Deployment options

The most autonomous agent will not automatically win. The most trustworthy autonomous agent may.

6. Multi-Agent Systems Will Become More Selective

The early temptation is to assign an agent to every subtask. Mature systems will learn where parallelism helps and where it merely multiplies cost, duplicated context, and coordination failure.

7. Harness Improvements Will Compound

Every traced failure can produce:

• A better tool
• A stronger policy
• A new verification gate
• A more useful skill
• A context-selection improvement
• A recovery strategy
• A regression test

That creates a reliability flywheel which model switching alone cannot provide.

8. Harnesses May Begin Adapting Themselves

Recent research is exploring systems that modify or optimize the executable harness around a fixed model. Test-Time Harness Evolution, for example, treats the harness itself as the object being adapted using execution traces rather than changing model weights. This remains early research, but it points toward a future where agents dynamically configure planning, tool use, verification, and recovery for different task distributions. Test-Time Harness Evolution The harness may eventually become an intelligent, adaptive system of its own.

Final Thought

The first era of AI-assisted development was about generation. The second era is about execution. Generating code requires intelligence. Reliably changing a production system requires:

• Context
• Tools
• Permissions
• Memory
• Verification
• Recovery
• Observability
• Human control

That entire surrounding system is the harness. Models will continue to improve. They will reason better, understand larger systems, and use tools more accurately. But intelligence without disciplined execution is only potential. The coding-agent winners will be the companies that convert that potential into software changes developers are willing to review, trust, and ship.

The model thinks.

The harness remembers, acts, checks, recovers, and earns trust.

That is why the next AI coding-agent war will not be fought only between models. It will be fought between the systems built around them.

Frequently Asked Questions

Is an agent harness just a system prompt?

No. A system prompt is one input to the model. A harness also manages context, tools, execution, permissions, memory, sessions, retries, verification, observability, and user interaction. Prompts guide behavior. Harnesses operationalize and enforce behavior.

Is MCP an agent harness?

No. MCP is a protocol for exposing tools, resources, and related capabilities. It can be an important part of a harness, but it does not automatically provide orchestration, sandboxing, context management, recovery, verification, or observability.

Can one harness support multiple models?

Yes. A provider-independent harness can normalize common concerns such as streaming, tool calls, errors, cancellation, context limits, and usage reporting while still preserving provider-specific strengths.

Does a better harness make the model irrelevant?

No. The model still determines major capabilities such as reasoning, code understanding, instruction following, and tool selection. The harness determines how reliably and safely those capabilities become real work.

What is the most important harness capability?

There is no single universal answer, but a strong verification and recovery loop is among the most important. Context and tools help an agent act. Verification tells it whether the action actually worked. Recovery determines whether failure becomes progress or repetition.

Should every coding agent use many subagents?

No. Subagents are valuable when work is genuinely separable. For small or tightly coupled changes, delegation can increase cost and coordination risk without improving quality.

Should companies build their own coding-agent harness?

Most teams should reuse an existing runtime for common orchestration and invest in the parts specific to their business:

• Domain context
• Tools
• Security policy
• Verification
• Observability
• Workflow integration
• Internal evaluations

Custom orchestration is justified when it solves a concrete requirement that existing runtimes cannot satisfy.

How should a company choose a coding agent?

Evaluate the complete model–harness pair on real internal tasks. Measure correctness, regression rate, verification, security, recovery, reviewability, tokens per solved task, cost per accepted change, and human intervention.

Key Takeaways

1. A coding agent is not simply a model with shell access.
2. The harness controls context, tools, execution, permissions, verification, memory, and recovery.
3. Recent studies show that changing the harness can materially change results and efficiency while keeping the model fixed.
4. Model quality remains essential, but model-only comparisons are incomplete.
5. Verification, sandboxing, and observability distinguish a production system from a demo.
6. The durable product moat may live in domain-specific context, tools, policies, evaluations, and reliability improvements.
7. The future belongs to strong model–harness combinations, not models operating in isolation.

Suggested Social Post

The AI coding-agent war is no longer just GPT vs Claude vs Gemini. The model writes code—but the harness decides:

• What context the model sees
• Which tools it can use
• What it is allowed to change
• How failures are recovered
• Whether the result is actually verified

The model sets the capability ceiling. The harness determines how reliably the system reaches it.

Read the full deep dive: The AI Coding Agent War Is No Longer About Models — It’s About the Harness

References

Product capabilities and linked documentation were checked on August 29, 2026. Product behavior may change after publication.

1. OpenAI — Codex as a platform: build on the open agent harness developers.openai.com/blog/codex-as-a-platform
2. OpenAI — Sandbox Agents developers.openai.com/api/docs/guides/agents/sandboxes
3. OpenAI — Shell + Skills + Compaction developers.openai.com/blog/skills-shell-tips
4. OpenAI — Build iterative repair loops with Codex developers.openai.com/cookbook/examples/codex/build_iterative_repair_loops_with_codex
5. OpenAI — Build an Agent Improvement Loop with Traces and Evals developers.openai.com/cookbook/examples/agents_sdk/agent_improvement_loop
6. OpenAI — Codex best practices: sandboxing and approvals developers.openai.com/codex/learn/best-practices
7. Anthropic — Claude Agent SDK overview code.claude.com/docs/en/agent-sdk/overview
8. Google — Gemini CLI policy engine geminicli.com/docs/reference/policy-engine
9. Google — Gemini CLI sandboxing geminicli.com/docs/cli/sandbox
10. Google — Gemini CLI checkpointing geminicli.com/docs/cli/checkpointing
11. Google — Gemini CLI subagents geminicli.com/docs/core/subagents
12. DeepSeek — DeepSeek Harness developer preview deepseek.com/harness/en
13. Model Context Protocol — Tools specification, 2026-07-28 modelcontextprotocol.io/specification/2026-07-28/server/tools
14. Lewis, S. — Same Model, Different Harness: Different Coding-Agent Results arXiv:2608.26218
15. Vats, N. and Golev, O. — The Scaffold Effect in Coding Agents arXiv:2607.22585
16. Yang, J. et al. — SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering arXiv:2405.15793
17. Nie, J. et al. — TTHE: Test-Time Harness Evolution arXiv:2607.08124

`