One-sentence Summary

Researchers from the Federal Institute of Goiás propose an operational definition of an agent harness by deriving necessary and sufficient conditions through a conceptual analysis, operationalizing them as an inclusion-and-exclusion test, and validating the definition against six real-world coding agent systems (Claude Code, Codex CLI, Aider, Cline, OpenHands, and SWE-agent) to establish a shared vocabulary that guides engineering practice and comparative evaluation.

Key Contributions

• A genealogy of the term "agent harness" is reconstructed from horse tack, through test harness and evaluation harness, establishing that the distinct agentic sense arises from control that acts at runtime rather than only after execution.
• A constitutive definition is proposed based on four necessary and sufficient conditions: agent loop, tool interface, context management, and control mechanisms; it is operationalized as an inclusion-and-exclusion test that separates an agent harness from neighboring concepts such as agent frameworks, SDKs, and orchestrators.
• The definition is applied to six real systems (Claude Code, Codex CLI, Aider, Cline, OpenHands, SWE-agent) and deliberate edge cases, demonstrating consistent classification and deriving a research agenda along four design-tension axes.

Introduction

Large language models have evolved from text generators into agents that reason, call tools, and act on codebases, creating a need for the surrounding infrastructure that ties a model to a task and controls its execution. In practice, this infrastructure is often called a harness, but the term is used inconsistently to refer to evaluation suites, orchestration layers, or entire products, making it difficult to compare systems, attribute design merit, or build trustworthy agents. The authors address this gap by constructing a precise, operational definition of an agent harness: a system that satisfies four constitutive conditions—agent loop, tool interface, context management, and control mechanisms—and they turn that definition into a reproducible inclusion/exclusion test that discriminates harnesses from neighboring concepts such as agent frameworks, SDKs, and evaluation harnesses.

Method

The authors define an agent harness as the runtime engineering layer that wraps one or more language models, transforming them into agents capable of accomplishing tasks within an external environment. This definition is established through four necessary conditions rather than specific examples. A system qualifies as an agent harness if and only if it instantiates these four elements at runtime, acting during the task execution rather than merely evaluating it afterward.

As shown in the figure below:

The anatomy of the agent harness places the model at the center as the engine. Surrounding it are the components required to make the engine drivable. The four core elements are strictly necessary. First, the agent loop interleaves reasoning, action, and observation. Without this loop, the system functions merely as a generator rather than an agent. Second, the tool interface allows the model to perceive and alter the external environment, such as editing files or running commands. Third, context management actively decides what enters and leaves the model's context window based on the task content, preventing information dilution. Fourth, control mechanisms provide limits, verification, and deterministic actions to ensure execution is trustworthy and contained, independent of the model's cooperation.

Beyond these core requirements, the architecture includes optional qualifiers that enhance robustness. These include memory for surviving across steps, a verifier to confirm task completion, retry logic with eventual model switching for transient failures, observability for auditing, guardrails for safety limits, and deterministic handlers for sensitive operations. These qualifiers are specializations of the core elements rather than new fundamental requirements.

While the core definition unifies the concept, specific implementations diverge based on design choices regarding real tensions. The authors organize these divergences along four design axes to map the research space.

As shown in the figure below:

The first axis contrasts autonomy versus control, balancing the agent's independent decision-making against safety and supervision. The second axis spans broad versus curated context, weighing the cost of actively selecting relevant information against the risk of diluting useful data in long contexts. The third axis distinguishes generalist versus specialized harnesses, addressing whether effective control mechanisms are reusable across domains or must be bespoke. The fourth axis ranges from open permission to containment, comparing agents running with user privileges against those operating in restricted sandboxes. These axes illustrate that while the concept of an agent harness is singular, its design space is wide and shaped by these fundamental trade-offs.

Experiment

The evaluation applies the inclusion and exclusion test to six real harnesses, confirming that all satisfy the core elements while correctly excluding systems like inline autocomplete and fixed pipelines that lack an adaptive loop or control. The analysis of design axes across these harnesses shows that control is the most differentiating dimension and a major open research front, while also revealing that the harness itself, rather than the underlying model, may become the crucial engineering factor. The study highlights the need for evaluation methods that isolate the harness's contribution from the model's performance.

The term harness has evolved from offline testing and evaluation scripts to a runtime layer that actively controls, limits, verifies, and corrects agent execution. This shift from observing to intervening during execution makes containment and control the most distinguishing design aspect, and it opens a methodological gap: current evaluation measures the combined model‑harness pair without isolating the harness’s own contribution. The agent engineering sense of harness operates during execution, unlike the previous two senses that only observe from the outside. Separating model from harness allows model switching to become a control mechanism, reducing application dependence on a single large model.

An agent harness is defined by satisfying four runtime conditions: a reasoning-action-observation loop, a tool interface to the environment, active context management, and at least one control mechanism independent of the model. Failing any condition places the system into a distinct neighboring category, such as a fixed pipeline, isolated model, naive wrapper, or untrusted demo. The control condition is especially discriminating, and the definition allows the harness's contribution to be separated from the model's. Without a runtime loop of reasoning, action, and observation, a system is a single-pass generator or fixed pipeline, not an agent. If no independent control mechanism exists, the system is a demo that trusts the model's word, lacking guarantees.

The agent harness is the only concept that satisfies all four criteria: a closed loop at runtime, action on an external environment, context and verification, and control. Every neighboring concept fails at least the runtime loop and control, while differing in whether they provide tools, act on the environment, or adapt to observations. The distinction reveals the harness as a distinct layer that assembles and runs a controlled agent, whereas frameworks, SDKs, plugins, eval harnesses, and orchestrators each occupy a different role in the agent ecosystem. The harness passes all four tests (T1–T4), while all five neighboring concepts fail at least T1 and T4. Agent frameworks compose agents but do not themselves execute the loop or control; they rely on embedded harnesses. IDE plugins lack the loop and do not act on the repository, closing no loop and providing no control. Eval harnesses and orchestrators provide tools (T2) but lack an adaptive, observation-driven loop (T1 and T3). Agent SDKs supply building blocks but do not assemble a runtime loop, making them raw material rather than a finished product.

All six real-world systems evaluated against the four core elements of an agent harness (T1–T4) satisfied each element, confirming they are agent harnesses. The form of control (T4) was the most varied element, with distinct mechanisms ranging from runtime guardrails and human approval to sandboxed execution and version-control auditability. This diversity makes control the primary design differentiator and a notably open front in the literature. All six systems satisfied T1 (agent loop), T2 (file and shell tools), T3 (context management), and T4 (control), confirming they are agent harnesses. The control element (T4) showed the greatest diversity, with distinct mechanisms such as runtime guardrails, permission modes, version-control auditability, human approval, sandboxing, and structured action interfaces. Control (T4) is the most distinguishing design feature and an area where the formal literature is least consolidated.

The analysis defines an agent harness as a runtime layer that must satisfy four conditions: a reasoning-action-observation loop, a tool interface, active context management, and at least one independent control mechanism, where the lack of any condition places a system into a distinct neighboring category (e.g., fixed pipeline, naive wrapper). By evaluating six real-world coding-agent systems against these criteria, the work confirms all qualify as harnesses, with the control dimension exhibiting the greatest diversity—spanning guardrails, sandboxing, human approval, and audit trails—and emerging as the most distinguishing design feature. This separation of model from harness also enables model switching as a control mechanism, reducing application dependence on a single large model.