One-sentence Summary

Researchers at Renmin University of China, IQuest Research, and Beihang University introduce Harness Scaling, an agentic reinforcement learning scaling dimension that extends training from a single fixed harness to diverse execution configurations by constructing a factorized harness space, selecting a compact training subset with balanced factorial coverage, and organizing cross-harness training with controlled variation, improving generalization to unseen white-box configurations and effective transfer to unseen external black-box harnesses across multiple benchmarks, model scales, and RL algorithms.

Key Contributions

• Introduces Harness Scaling, a scaling dimension for agentic RL that trains policies across diverse execution harness configurations rather than a single fixed harness, addressing specialization and limited transfer.
• Develops a structured framework that factorizes harness variation, selects a compact training subset with balanced factorial coverage, and assigns structurally nearby configurations to the same task during cross-harness training.
• Across benchmarks, model scales, and RL algorithms, Harness Scaling improves generalization to unseen white-box harnesses and transfers effectively to complex external black-box harnesses, outperforming training through a single complex black-box harness.

Introduction

Agent harnesses shape how language-model agents use tools, manage context, and recover from errors, making them central to long-horizon agentic systems. Prior work mainly scales the tasks and environments used for reinforcement learning, or trains policies inside one fixed harness, which can cause overfitting to a specific execution setup and poor transfer to new harnesses. The authors introduce Harness Scaling, treating the harness as a trainable variation dimension by factorizing it into system prompt, tool interface, observation, context management, and workflow. They leverage orthogonal design to build a compact set of diverse harness configurations and pair each task with structurally nearby configurations during RL. This approach improves generalization to unseen white-box harnesses and transfers effectively to external black-box harnesses.

Method

The authors propose Harness Scaling, a methodology that treats the execution harness as a new scaling axis to improve policy generalization. Rather than relying on a single monolithic execution environment, the framework systematically explores diverse execution configurations to train more robust policies.

As illustrated in the framework diagram, the approach encompasses three core stages: conceptualizing the harness as a scaling axis, constructing a structured combinatorial harness space, and executing cross-harness reinforcement learning to achieve generalization across unseen configurations and complex real-world harnesses.

To study this systematically, the authors formalize the space of execution configurations by factorizing the harness into five distinct components: system prompt ($S$S), tool interface ($T$T), observation ($O$O), context management ($C$C), and workflow ($W$W). Each component is instantiated through a family of specific transformations applied to a base realization. A complete harness configuration $h$h selects one realization from each component set, defining the factorized harness space as:

$$\mathcal{H} = \mathcal{S} \times \mathcal{T} \times \mathcal{O} \times \mathcal{C} \times \mathcal{W}$$H=S×T×O×C×W

The product structure of this space induces a natural notion of structural proximity, measured by the Hamming distance $d_{\mathrm{H}}(h_i, h_j)$dH​(hi​,hj​), which counts the number of component dimensions on which two configurations differ. The generalization objective focuses on two complementary forms: in-space combinatorial generalization to held-out configurations within $\mathcal{H}$H, and out-of-space generalization to complex real-world black-box harnesses beyond the constructed design space.

Because the factorized harness space grows combinatorially, exhaustive training is computationally prohibitive and introduces redundancy. To address this, the authors construct a compact training set that preserves broad factorial coverage under a fixed optimization budget. They treat the harness space as a factorial design and enforce strength-2 balance. This requires component balance, ensuring each realization is represented evenly, and pairwise combination balance, where every pair of realizations from any two components appears equally often in the selected set. The authors instantiate a strength-2 orthogonal design over the component realizations to directly construct the training harness set $\mathcal{H}_{\mathrm{tr}} \subset \mathcal{H}$Htr​⊂H. This design satisfies the Rao bound and ensures that while low-order projections are perfectly balanced, many complete harness combinations remain unseen, thereby enabling the evaluation of in-space combinatorial generalization.

The training procedure exposes the policy to multiple execution configurations for the same underlying task through a locality-aware assignment strategy and multi-harness policy optimization. For the cross-harness task assignment, the authors assign a small group of structurally related configurations to each task. For a given task $x$x, an anchor harness $h_x$hx​ is sampled uniformly from the training set $\mathcal{H}_{\mathrm{tr}}$Htr​. The remaining $K-1$K−1 configurations are selected as those with the smallest Hamming distance to the anchor. This yields a task-specific harness set:

$$\mathcal{H}_x = \{h_x, h_x^{(1)}, \dots, h_x^{(K-1)}\}$$Hx​={hx​,hx(1)​,…,hx(K−1)​}

This approach combines local cross-harness variation with broad global coverage, as different tasks are distributed across different regions of the training space.

During multi-harness policy optimization, each task-harness instance $(x, h)$(x,h) is executed to produce a trajectory $\tau$τ and a corresponding reward. The $K$K harnesses assigned to the same task are utilized over successive training steps. For a policy-update batch $\mathcal{B}$B induced by the task-harness training distribution, the authors optimize the standard clipped policy objective:

$$\mathcal{L}(\theta) = \frac{1}{N_{\mathcal{B}}} \sum_{(x, h, \tau) \in \mathcal{B}} \sum_{t=1}^{|\tau|} M_t \min\left(\rho_t(\theta) A_t, \operatorname{clip}\left(\rho_t(\theta), 1-\epsilon, 1+\epsilon\right) A_t\right)$$L(θ)=NB​1​(x,h,τ)∈B∑​t=1∑∣τ∣​Mt​min(ρt​(θ)At​,clip(ρt​(θ),1−ϵ,1+ϵ)At​)

where $N_{\mathcal{B}}$NB​ is the total number of valid policy-generated tokens, $\rho_t(\theta)$ρt​(θ) is the probability ratio, $M_t$Mt​ is a masking variable, and $A_t$At​ is the advantage computed by the underlying reinforcement learning algorithm. This cross-harness training procedure is seamlessly applied to both PPO and GRPO, allowing the policy to learn robust behaviors that generalize beyond any single harness configuration.

Experiment

The experiments compare Harness Scaling with single-harness and black-box baselines across multiple agent benchmarks, testing in-space generalization to unseen harness configurations and transfer to external black-box harnesses. Results show that training over structured, diverse harness configurations improves robustness to execution settings without requiring exposure to every combination, and this benefit persists across model sizes, RL algorithms, and external harnesses. Ablations indicate that harness selection, per-task harness diversity, and locality-aware assignment are important, while behavioral analysis shows gains in state tracking, workflow compliance, and error recovery rather than overfitting to the base harness.

Harness scaling improves pass@1 results across held-out configurations and the external black-box harness compared with single-harness training. The benefit appears for both smaller and larger backbones and for GRPO and PPO, indicating robustness across model sizes and optimization methods. The trend also suggests that training over diverse structured harnesses supports better cross-harness transfer than direct training through one complex harness. Harness scaling raises average performance on held-out harness configurations for both evaluated model scales. The improvement holds under GRPO and PPO rather than depending on one optimizer. The scaled harness approach remains effective on the external black-box harness, where it outperforms direct black-box training baselines.

On Claude Code, models trained with Harness Scaling achieve the highest average Pass@1 for both Qwen model scales, surpassing black-box SFT and black-box RL baselines trained through OpenClaw. The result indicates that structured training over diverse white-box harness configurations transfers more effectively to an unseen black-box harness than direct training through a single complex black-box harness. Harness Scaling improves transfer to the unseen black-box harness at both model scales. Black-box SFT yields higher average Pass@1 than black-box RL in this transfer setting.

Structured selection of the 18-harness training set outperforms a randomly selected 18-harness set on both benchmarks under both evaluation settings. Locality-aware assignment also improves over random assignment, supporting the combination of broad harness coverage with controlled per-task variation. Structured 18-harness selection yields higher In-Space and Claude Code scores than Random-18 on ClawGym-Bench and PinchBench. Locality-aware assignment improves over random assignment across both benchmarks and evaluation settings.

For Qwen3-30B-A3B-Instruct trained with GRPO, scaling across harnesses leaves base-harness performance nearly unchanged while increasing in-space benchmark and Claude Code scores. The larger gains on alternative harnesses, compared with the slight base-harness decline, indicate better robustness to execution variation without major specialization loss. Base-harness performance changes only slightly after scaling. In-space and Claude Code performance improve under scaling, with the in-space gain especially large.

Experiments evaluate pass@1 performance across held-out harness configurations, an external black-box harness, and benchmark suites using Qwen backbones trained with GRPO and PPO. Harness scaling consistently improves transfer to unseen harnesses while preserving base-harness performance, indicating robustness across model scales and optimization methods. Structured harness selection with locality-aware assignment also outperforms random selection, suggesting that broad but controlled exposure to execution environments supports better cross-harness generalization.