One-sentence Summary

Researchers from Washington University in St. Louis, Google Cloud, and University of North Carolina at Chapel Hill propose Environment Harness (EnvHarness), a programmable plug-in layer that reshapes static environments without altering underlying logic, and EnvRigger, which observes black-box policy trajectories to synthesize and validate components, achieving up to a $9.0$9.0-point improvement on held-out instances with $9.8\%$9.8% fewer execution steps and enabling improved reinforcement-learning co-evolution across four domains.

Key Contributions

• EnvHarness is a programmable layer that wraps a static environment through its standard reset/step interface and uses plug-in components Stage, Contract, and Chain to reshape initial states, agent-environment interaction, and composite tasks without modifying the underlying environment logic or its original verifier.
• EnvRigger automates policy-conditioned environment customization by treating the target policy as a black box, diagnosing weaknesses from execution trajectories, and iteratively revising candidate components until fresh rollouts confirm success, ensuring each new environment targets the policy’s specific flaws.
• Across five benchmarks in four domains, EnvHarness outperforms original environments and domain-specific generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. It also provides a stronger optimization signal for reinforcement learning, enabling continuous policy-environment co-evolution.

Introduction

As LLMs become autonomous agents, their learning increasingly depends on interactive environments for web navigation, code tasks, and embodied control. Manually building these environments is expensive and produces rigid, static benchmarks that do not adapt to a specific agent’s weaknesses or provide new challenges after mastery. Automated generation scales environment creation, but existing pipelines are domain-specific and often require costly over-generation and filtering because LLM-generated verifiers are hard to trust. The authors address these issues with EnvHarness, a programmable layer that wraps an existing static environment through its standard reset/step interface using three plug-in components: Stage, Contract, and Chain. This approach customizes initial states, allowed actions and observations, and composite task horizons without modifying the underlying environment, so it is domain-agnostic and retains the original human-built verifiers. To make customization automatic, they introduce EnvRigger, which diagnoses a policy’s behavioral weaknesses from execution trajectories and iteratively creates and tests EnvHarness components to target those weaknesses.

Method

The authors introduce EnvHarness, a programmable layer that wraps an existing, static environment to transform it into a customizable one without modifying the underlying simulator backend. Formally, an environment is modeled as a tuple $E = (S, \mathcal{A}, O, T, R, s_0)$E=(S,A,O,T,R,s0​). An EnvHarness component applies an environment-agnostic transformation $w$w to reshape the environment strictly at the interface level:

$$E' = w(E), \quad E' = (\mathcal{S}', \mathcal{A}', \mathcal{O}', T', R', s_0').$$E′=w(E),E′=(S′,A′,O′,T′,R′,s0′​).

This approach preserves the ground-truth evaluation logic, ensuring the original verifier can still score the episode.

The framework relies on three concrete types of modular plug-in components to tailor the environment for specific training needs.

As shown in the figure above, these components override standard environment interface methods such as reset or step:

1. Stage: Specified by a sequence of state-manipulation actions $\delta = (a_1, \dots, a_k)$δ=(a1​,…,ak​), a Stage customizes the agent's starting point by applying these actions to the initial state $s_0$s0​. This allows the system to introduce obstacles or complete early subgoals in advance.
2. Contract: Defined by a triplet of transformation maps $\boldsymbol{r} = (f_A, f_T, f_O)$r=(fA​,fT​,fO​), a Contract rewrites the action space, transition dynamics, and observation space. These transformations can enforce action preconditions, mask observations, or attach structured feedback to steer agent learning.
3. Chain: Specified by a pair $\ell = (E_{\text{ext}}, g)$ℓ=(Eext​,g), a Chain extends the environment by combining the original environment with an additional environment $E_{\text{ext}}$Eext​ using composition logic $g$g. This allows for concatenating, interleaving, or branching environments dynamically.

Because all components share a standard interface, they compose freely, though the nesting order determines the final environment construction.

To automatically generate these task-policy-conditioned transformations, the authors introduce EnvRigger. Given a base environment $E$E, a target policy agent $\pi$π, and a task $t$t, EnvRigger realizes the map $\mathcal{H}$H:

$$E' = \mathcal{H}(E, t; \pi) = (w_k \circ w_{k-1} \circ \dots \circ w_1)(E),$$E′=H(E,t;π)=(wk​∘wk−1​∘⋯∘w1​)(E),

where each $w_i$wi​ is a customized EnvHarness component designed to expose the critical weaknesses of $\pi$π on task $t$t.

Refer to the framework diagram for the complete workflow, in which EnvRigger operates systematically through four distinct stages:

• Observe: The system runs the policy $\pi$π on the base task in the current environment to collect and analyze a batch of rollout trajectories. Failures expose specific weaknesses, while successes define the boundaries of these flaws.
• Diagnose: EnvRigger analyzes the trajectories to identify root causes of observed behaviors, such as repetitive action loops or failures in parsing long observations. It determines the customization direction, deciding whether to scaffold missing steps for a struggling policy or inject more challenging scenarios if the policy achieves a perfect success rate.
• Write: Based on the diagnosis, EnvRigger synthesizes one or more EnvHarness components to target the identified flaws. A single flaw may require combining multiple components, such as a Stage and a Contract, emitted together as a candidate set.
• Validate: To evaluate the candidate components, EnvRigger wraps the current environment with them to instantiate $E'$E′ and runs fresh rollouts. Based on trajectory metrics like success rate and failure distribution, the system decides to accept the candidate, reject it, or refine it. If refinement is required, the trajectories and scaling feedback flow back into the Write stage, repeating this loop until a candidate is accepted or the revision budget is exhausted. All accepted components are ultimately added to the EnvHarness.

Experiment

EnvHarness uses a diagnostic write-and-validate loop to generate customized training environments that target specific policy weaknesses, and skills extracted from these environments consistently improve performance over static and domain-specific generation methods across five diverse benchmarks. The approach generalizes across model families and learning paradigms, including online reinforcement learning, scales efficiently by co-evolving with the policy, and can incorporate explicit user constraints to target specific behaviors.

EnvHarness treats a static environment as the base system and adds an external customization layer for states, rules, and observations, analogous to how agent harnesses add capabilities to a frozen LLM. Generated environments outperform original environments in targeted settings, and EnvHarness skills exceed real environment skills across multiple LLM backbones with gains that remain consistent across weak and strong policies. The loop can also accept user-defined constraints to target specific weaknesses and distill focused skills such as verification-driven development. EnvHarness keeps the base environment fixed and adds customization externally, producing a customized environment instead of altering the core interaction logic. Across diverse LLM backbones, EnvHarness skills outperform real environment skills by a consistent margin, and explicit constraints can target weaknesses such as submitting patches without running tests.

Skills extracted from EnvHarness-customized environments improve over both no-skill and original-environment skill baselines across the reported ALFWorld and WebArena settings. The largest gains over original environments appear in out-of-distribution ALFWorld and WebArena shop admin. Benchmark-specific generation baselines are limited to one domain, while EnvHarness applies to both and raises average performance. EnvHarness skills improve every reported ALFWorld and WebArena metric relative to the no-skill baseline. Original environment skills deliver mixed results and fall below the no-skill baseline in WebArena Reddit and GitLab. EnvHarness gains over original environments are strongest for ALFWorld out-of-distribution at 9.0 points and WebArena shop admin at 6.2 points.

Skills extracted from EnvHarness-customized environments consistently outperform skills from original static environments across all reported SWE-bench Verified, OfficeQA, and SpreadsheetBench metrics. The largest gains are in SWE-bench trajectory efficiency and SpreadsheetBench pass rate, while original environment skills can reduce SpreadsheetBench pass rate below the no-skill baseline and lengthen SWE-bench trajectories. EnvHarness also applies across domains where the benchmark-specific source cannot. EnvHarness environment skills outperform original environment skills on every reported metric, with the strongest gains in SWE-bench average step reduction and SpreadsheetBench pass rate. Skills from original static environments can be counterproductive: they lower SpreadsheetBench pass rate below the no-skill baseline and increase SWE-bench average steps. A benchmark-specific SWE source is unavailable for OfficeQA and SpreadsheetBench, whereas EnvHarness skills are evaluated across all three domains.

Reinforcement learning on environments reshaped by EnvHarness generally outperforms training on the original static environments. The reshaped environments improve ALFWorld in-distribution success and average success, as well as WebShop score and success rate, with only a negligible decrease on ALFWorld held-out success. ALFWorld in-distribution success rate rises from 81.4 to 87.9, and average ALFWorld success improves from 85.5 to 88.4 when training on reshaped environments. WebShop score increases from 75.6 to 79.2 and success rate increases from 66.0 to 67.4, while ALFWorld held-out success stays nearly unchanged.

On long-horizon environments, combining stage/contract skills with chain pairing achieves the highest success rate while also reducing average steps compared with baselines. Stage/contract skills alone mainly improve success rate, whereas chain pairing alone mainly improves efficiency by lowering average steps. The combined configuration shows these benefits are complementary. Combining stage/contract and chain skills yields the strongest success rate across the evaluated conditions. Chain pairing substantially lowers average steps relative to no-skill and original environment settings. Stage/contract skills alone provide a clear success rate gain, with a more modest step reduction than chain-based or combined configurations.

EnvHarness adds an external customization layer to static environments, generating tailored environments that improve agent skills without altering the core interaction logic. Across diverse LLM backbones and benchmarks including ALFWorld, WebArena, SWE-bench, OfficeQA, and SpreadsheetBench, skills from EnvHarness-customized environments consistently outperform those from original static environments, with particularly strong gains in out-of-distribution settings and efficiency metrics. User-defined constraints can target specific weaknesses, reinforcement learning on reshaped environments yields better performance, and combining stage/contract skills with chain pairing maximizes success rates while reducing steps in long-horizon tasks.