The Most Comprehensive Survey of Agent Harness Engineering Revealed

We share a systematic and engineering‑focused review of the paper Agent Harness Engineering: A Survey , a joint effort by CMU, Yale, JHU, Virginia Tech, Amazon and others.

The paper argues that while academic research has long emphasized model improvements, real‑world agent failures often stem from inadequate surrounding systems rather than model intelligence.

Benchmark results illustrate this point: modifying only the tool interface and harness (no model change) can yield up to a 10× boost on certain benchmarks; a GPT‑5.2‑Codex agent improved its success rate from 52.8% to 66.5% on Terminal‑Bench 2.0 by redesigning system prompts, adding context injection hooks, and self‑validation; Meta‑Harness achieved 76.4% on the same benchmark by automatically optimizing the harness, surpassing handcrafted solutions.

同一个模型，换一套执行外壳，表现可以完全不一样。

These numbers, while dependent on experimental settings, highlight that the execution shell surrounding a model can dramatically affect performance.

Three Evolutionary Stages of Agent Engineering

Stage 1 – Prompt Engineering: The focus is on crafting system prompts, few‑shot examples, and step‑by‑step reasoning to coax the model.

Stage 2 – Context Engineering: As agents tackle longer tasks, the challenge shifts to deciding what information enters the short‑term context, how to retrieve memories, compress tool results, and manage window overflow.

Stage 3 – Harness Engineering: Once models can handle complex tasks, bottlenecks move outside the model: state maintenance, tool orchestration, permission control, feedback injection, progress verification, tracing, and failure recovery.

In short, Prompt Engineering asks “how to talk to the model,” Context Engineering asks “what the model should see,” and Harness Engineering asks “how to make the model act reliably in the real world.”

ETCLOVG: The Seven‑Layer Harness Framework

The survey introduces a seven‑layer classification called ETCLOVG :

Execution : Where does the agent run? Local, container, browser, desktop, remote sandbox?

Tooling : How are tools described, discovered, invoked, and guarded against misuse?

Context : Management of short‑term context, session state, and long‑term memory.

Lifecycle : Single‑round vs. multi‑round execution, division of labor among planner, executor, reviewer.

Observability : Tracing model calls, tool invocations, retrievals, errors, retries, token cost, latency.

Verification : Determining whether results are correct and diagnosing failures (model, tool, context, or test environment).

Governance : Defining permissions, approvals, and audit trails for actions such as code changes, API calls, or data access.

All seven layers together constitute a production‑grade agent system.

Why Observability and Governance Matter

The paper emphasizes that observability and governance are not peripheral add‑ons but essential layers. Agents can execute shell commands, modify code, access databases, and send emails; without visibility into actions and clear permission boundaries, failures become opaque and unsafe.

Trace‑native evaluation is advocated: record model outputs, tool calls, environment changes, context snapshots, errors, retries, token usage, latency, and cost. Then assess (1) result correctness, (2) reasonableness of the execution path, and (3) trustworthiness of the evaluator itself.

Cross‑Layer Trade‑offs

The survey identifies three major contradictions:

Cost‑Quality‑Speed triangle: stronger safety (sandboxing, fine‑grained permissions, comprehensive tracing) increases cost and latency.

Capability‑Control tension: richer toolsets and longer memory improve utility but raise risks of tool misuse, prompt injection, privacy leaks, and stale information.

Harness coupling: layers are interdependent; tool descriptions affect context windows, execution environments influence evaluation outcomes, and observability data must include identity and permission context for governance.

Thus, optimizing one layer (e.g., prompt) without considering its impact on others can lead to suboptimal system behavior.

From Framework to Platform

Early work focused on quickly assembling an agent loop (framework). The current competition centers on building a full production platform that provides durable workspaces, managed sandboxes, identity management, billing, observability, evaluation, governance, and human handoff.

Success in the platform era depends on robust execution environments, clear tool protocols, stable context handling, effective tracing, realistic verification, and controllable permissions.

Future Direction: Less Scaffolding

While adding control layers can improve reliability, they may become unnecessary as models grow stronger. The paper cites an Anthropic example where context resets that helped older models were removed for newer ones, reducing cost without harming quality.

The key insight is that good harness engineering knows when to add controls and when to remove them.

Conclusion

Agent development has progressed from prompt engineering to context engineering, and now to harness engineering. The next competitive frontier lies not in model capability alone but in the engineering shell that enables agents to act safely and reliably in real environments.