From Prompt to Context to Harness: Unpacking the Three Paradigm Shifts in Agent Engineering

Survey Overview

Agent Harness Engineering: A Survey, a joint effort by researchers from CMU, Yale, JHU, Virginia Tech and Amazon, provides the most systematic engineering‑focused overview of the layers that surround a running LLM‑based agent. The paper’s homepage is https://picrew.github.io/LLM-Harness/.

Benchmark Evidence

Modifying only the harness—changing editor tool formats, adding context‑injection middleware, or inserting verification hooks—can produce up to a ten‑fold improvement on coding benchmarks. A fixed GPT‑5.2‑Codex agent increased from 52.8 % to 66.5 % on Terminal‑Bench 2.0 after redesigning its system prompt, adding middleware‑based context injection, and self‑verification hooks. Meta‑Harness achieved 76.4 % on the same benchmark, surpassing hand‑crafted solutions.

Key Insight

The same model, wrapped in a different execution shell, can behave completely differently. In many deployments the failure is not an insufficient model but a weak surrounding system (tool interface, context manager, sandbox, verification, or permission layer).

Paradigm Shifts

Prompt Engineering (2022) : Crafting system prompts, few‑shot examples, and step‑by‑step reasoning to make a single input work.

Context Engineering (2023‑2024) : Deciding what information the model should see at each step, how to retrieve memories, compress tool results, and handle window overflow for long‑running tasks.

Harness Engineering (2025‑2026) : Managing execution environment, tool orchestration, state persistence, observability, verification, and security governance so that the model can act reliably in the real world.

ETCLOVG Seven‑Layer Framework

Execution : Where does the agent run? (local, container, browser, remote sandbox, etc.)

Tooling : How are tools described, discovered, invoked, and protected from misuse?

Context : Management of short‑term context, conversation state, and long‑term memory.

Lifecycle : Single‑round vs. multi‑round loops, planner‑executor‑reviewer division.

Observability : Tracing model calls, tool calls, errors, retries, token cost, latency.

Verification : Checking result correctness and diagnosing whether failures stem from model, tool, context, or test environment.

Governance : Permission checks, identity, audit, and approval workflows.

All seven layers together constitute a system capable of handling long‑duration tasks.

Why Observability and Governance Stand Alone

In production, agents invoke tools, run shell commands, modify code, and access databases. Without full observability you cannot diagnose failures, and without governance you cannot safely deploy agents. Observability records every model call, tool call, retrieval, error, retry, token usage and latency; governance enforces who may act, what actions are permitted, and provides audit trails.

Evaluation Shift

Traditional benchmarks report only final success rates. The survey advocates trace‑native evaluation : record the full execution trace (model outputs, tool calls, tool returns, environment state changes, context snapshots, errors, retries, token usage, latency, and cost) and then assess three aspects—result correctness, path reasonableness, and evaluator trustworthiness. This moves evaluation from a ranking mindset to a quality‑control mindset.

Cross‑Layer Trade‑offs

Cost‑quality‑speed triangle: stronger sandboxing, finer permissions, richer traces increase cost and latency.

Capability vs. control: richer toolsets and longer memory improve usefulness but raise prompt‑injection risk and privacy concerns.

Harness coupling: changes in one layer (e.g., tool description) affect others (context window, model behavior, evaluation).

From Framework to Platform

Frameworks abstract agent, tool, memory, and loop. Platforms must provide a durable workspace, managed sandbox, identity, billing, observability, evaluation, governance, and human hand‑off. The competition will shift from model performance to the quality of the entire harness stack.

Future Outlook

Developers should no longer ask only “which model is stronger?” but also consider where it runs, how its tool interfaces are designed, whether context drifts, whether state can be recovered across rounds, whether traces enable failure diagnosis, whether verifiers exist, and whether permissions and audits are closed‑loop. Prompt engineering awakens the model, context engineering shows it the right information, and harness engineering makes it act reliably in the real world.

Illustrative Figures

Code example

同一个模型，换一套执行外壳，表现可以完全不一样。