The Most Comprehensive Survey of Agent Harness Engineering

The article introduces the survey paper "Agent Harness Engineering: A Survey" authored by researchers from CMU, Yale, JHU, Virginia Tech, Amazon and others, which systematically analyzes the engineering systems surrounding agents beyond the model itself.

It argues that academic research has focused on model capabilities, but when agents tackle long‑running tasks in real environments, failures often stem from inadequate system support rather than model intelligence.

Benchmark results cited include: (1) modifying only the tool interface and surrounding harness can improve performance up to ten‑fold; (2) a fixed GPT‑5.2‑Codex agent gains from redesigned system prompts, middleware context injection, and self‑validation hooks, raising Terminal‑Bench 2.0 success from 52.8% to 66.5%; (3) Meta‑Harness automatically optimizes the harness to achieve 76.4% on the same benchmark, surpassing handcrafted solutions.

The same model can behave completely differently when wrapped with a different execution shell.

The survey proposes a three‑stage evolution of agent engineering (2022‑2026): Prompt Engineering (optimizing system prompts and few‑shot examples), Context Engineering (managing what the model sees, memory retrieval, tool result compression, and window overflow), and Harness Engineering (handling state maintenance, tool orchestration, permission control, feedback injection, verification, and tracing).

To formalize Harness Engineering, the authors introduce the seven‑layer ETCLOVG framework: Execution, Tooling, Context, Lifecycle, Observability, Verification, and Governance. Each layer addresses a specific responsibility, from where the agent runs (local, container, sandbox) to how permissions and audits are enforced.

Observability and Governance are highlighted as independent layers rather than peripheral features. Without proper tracing, failures are opaque; without governance, even successful runs may be unsafe.

The paper advocates a shift in evaluation methodology: instead of measuring only final success rates, evaluations should be trace‑native, recording model outputs, tool calls, environment state changes, errors, retries, token usage, latency, and costs, then assessing result correctness, execution path reasonableness, and evaluator trustworthiness.

It also notes a trend from Agent Frameworks (focused on isolated loops) to Agent Platforms that provide durable workspaces, managed sandboxes, identity, billing, observability, evaluation, governance, and human handoff. Competitive advantage will increasingly depend on the quality of the entire harness stack rather than just model performance.

Finally, the survey warns that adding more control layers is not always beneficial; as models become stronger, some wrappers become unnecessary and may even hinder performance. Effective harness engineering requires knowing when to add or remove controls.