The Most Comprehensive Survey on Agent Harness Engineering Revealed

01. Model Upgrade Is Not the Most Effective Agent Improvement

The paper observes that academic research has long focused on model capabilities, but when agents tackle long‑running tasks with real tools, failures often stem from inadequate system engineering rather than model intelligence.

Empirical results show that modest changes to the harness can yield large gains: altering tool‑formatting and surrounding harness without touching the model achieved up to 10× improvement on coding benchmarks; a fixed GPT‑5.2‑Codex agent improved from 52.8% to 66.5% on Terminal‑Bench 2.0 by adding system prompts, middleware context injection, and self‑validation hooks; Meta‑Harness reached 76.4% on Terminal‑Bench‑2 , surpassing handcrafted solutions.

These numbers illustrate that

the same model can behave dramatically differently with a different execution shell

. Many teams still blame “model weakness”, yet the real bottleneck often lies in tool interfaces, context management, sandboxing, verification, or permission systems.

02. Three Migration Stages of Agent Engineering

The survey proposes a timeline from 2022 to 2026, dividing agent development into three phases (see Figure 1):

Prompt Engineering : crafting system prompts, few‑shot examples, and step‑by‑step reasoning.

Context Engineering : deciding what information enters the short‑term context, how to retrieve memories, compress tool results, and handle window overflow for long‑running tasks.

Harness Engineering : managing state, tool orchestration, permission control, feedback injection, progress verification, and trace logging.

Prompt Engineering answers “how to talk to the model”, Context Engineering answers “what the model should see”, and Harness Engineering answers “how to make the model work reliably in the real world”.

03. What a Harness Actually Contains

The authors introduce the seven‑layer ETCLOVG framework (Execution, Tooling, Context, Lifecycle, Observability, Verification, Governance). Each layer is described as follows:

Execution : where the agent runs (local, container, browser, remote sandbox) and its boundaries.

Tooling : description, discovery, invocation, and safeguards against inappropriate tool selection.

Context : short‑term context, session state, and long‑term memory management.

Lifecycle : single‑round vs. multi‑round loops, division of labor among planner, executor, reviewer.

Observability : tracing model calls, tool calls, retrievals, errors, retries, token cost, and latency.

Verification : checking result correctness and diagnosing whether failures stem from model, tool, context, or test environment.

Governance : permission checks, audit trails, and approval workflows for actions such as code changes, API calls, or data access.

Only by integrating all seven layers can an agent reliably execute long‑duration tasks.

04. Observability and Governance as First‑Class Layers

Previous frameworks treated logging, monitoring, and access control as afterthoughts. The survey argues that in production these are essential: agents may invoke shells, run commands, modify code, or access databases, and without observability you cannot diagnose failures, while without governance you cannot safely deploy agents.

05. Rethinking Agent Evaluation

Traditional benchmarks report a single success rate, but the paper shows that identical success rates can mask vastly different system qualities (e.g., heavy retry costs, unsafe paths, benchmark‑specific hacks). Therefore, evaluation should be trace‑native : record the full execution trace—including model outputs, tool calls, environment state changes, errors, retries, token usage, latency, and cost—then assess three aspects: result correctness, reasonableness of the execution path, and trustworthiness of the evaluator itself.

06. The Cost‑Quality‑Speed Triangle and Harness Coupling

Improving safety requires stronger sandboxes, finer‑grained permissions, and richer traces, which increase cost and latency. Adding more tools expands capability but raises the risk of wrong tool selection and prompt injection. Each harness layer is coupled: tool descriptions consume context windows; execution environments affect evaluation outcomes; observability traces must capture identity and permission state to serve as governance evidence.

07. From Framework to Platform

The survey predicts a shift: early work focused on isolated abstractions (agent, tool, memory, loop). Future platforms must provide end‑to‑end production capabilities—durable workspaces, managed sandboxes, identity, billing, observability, evaluation, governance, and human handoff. Competition will therefore move from model performance to the quality of the entire harness stack.

08. When to Remove Harness Layers

As models become stronger, some harness components become unnecessary. The paper cites an Anthropic example where a context reset that helped older models could be dropped for newer ones, reducing cost without harming quality. Good harness design knows both when to add controls and when to discard them.

09. Conclusion: The Next Competition Is the Engineering Shell

In summary, the next frontier for agents is not larger models but the robustness of the surrounding engineering shell. Developers should ask not only “which model is stronger?” but also “where does it run?”, “are tool interfaces agent‑aware?”, “does context drift?”, “can state be recovered across rounds?”, “is failure traceable?”, “are results verified?”, and “are permissions and audits closed‑loop?”. Prompt Engineering awakens the model, Context Engineering shows it the right information, and Harness Engineering ensures it acts reliably in the real world.