HyperAI

L'évolution des harnais à partir d'une base commune présente localité, complémentarité et non-stationnarité : les variantes réparent différentes défaillances et leur utilité change à mesure que la politique apprend. Plutôt que de chercher un harnais universellement supérieur, nous introduisons Harness-S pour remédier sélectivement aux échecs d'un système d'exécution de base à l'aide de harnais frères et pour transférer les capacités qui en résultent dans une politique unique qui reste efficace avec le harnais d'origine. L'évolution directionnelle des harnais développe des harnais frères spécialisés selon des directions de réparation distinctes, formant un pool de harnais frères réutilisable qui demeure fixe pendant l'entraînement de la politique. L'entraînement de secours complémentaire par harnais n'invoque ce pool que lorsque toutes les tentatives initiales sous le harnais de base échouent. La même politique génère de nouvelles trajectoires sous des harnais auxiliaires sélectionnés, et ces trajectoires remplacent le groupe en échec pour l'apprentissage par renforcement. Un routeur adaptatif de harnais sélectionne les k meilleures variantes à l'aide de profils de comportement des harnais et d'un vote entre plusieurs LLM. Le réancrage des profils, sensible à la politique, actualise ces informations au fur et à mesure de l'apprentissage de la politique. Sur AppWorld, SWE-Gym et DBBench, la politique entraînée surpasse nettement l'entraînement fondé uniquement sur la politique lorsqu'elle est ramenée au harnais de base d'origine. Les analyses comportementales et les cas tenus hors échantillon confirment le transfert des comportements induits par les harnais vers la politique elle-même. La réutilisation du routeur pour l'inférence de rang 1 apporte des gains supplémentaires à nombre de déploiements d'exécuteur égal.

One-sentence Summary

Researchers from Shanghai Jiao Tong University, Xiaohongshu Inc., and other institutions propose Harness-S, which learns from fixed sibling harnesses through directional evolution, complementary rescue training, and an adaptive LLM-voting router, transferring failure-repair capabilities into a single policy and improving AppWorld, SWE-Gym, and DBBench performance as well as Rank-1 inference.

Key Contributions

The paper introduces Harness-S, a framework that uses directional harness evolution to build specialized sibling harnesses along distinct repair directions and stores them in a reusable sibling harness pool that remains fixed during policy training.
The paper presents Complementary Harness Rescue Training with an Adaptive Harness Router, which invokes the sibling pool only when all initial base-harness attempts fail and selects Top-k variants using Harness Behavior Profiles and voting across multiple LLMs; policy-aware profile re-anchoring refreshes this evidence as the policy learns.
Experiments across AppWorld, SWE-Gym, and DBBench show that the trained policy substantially outperforms policy-only training when evaluated under the original base harness; behavioral analyses and held-out cases support capability transfer, and Rank-1 routing provides additional gains at the same executor rollout count.

Introduction

The authors study how an agent’s execution harness, including prompts, tools, memory, and execution rules, shapes policy behavior even when model weights are fixed. Prior work automates prompt, workflow, and harness optimization, but harness edits often show locality, complementarity, and nonstationarity: repairs fix some failures while causing regressions, sibling harnesses can recover each other’s errors, and harness utility changes as the policy learns. These properties make it difficult to combine local repairs into one universal harness and make coupled harness-policy evolution costly, with gains that may not transfer back to the original harness. The authors introduce Harness-S, which evolves specialized sibling harnesses along distinct repair directions, uses an adaptive router to rescue failed base-harness attempts during training, and transfers the resulting capabilities into a single policy that remains effective under the original base harness.

Method

The authors propose Harness-S, a framework that leverages selective rescue of base harness failures to transfer useful behaviors from specialized sibling harnesses into a shared policy. This approach enables the policy to learn from alternative execution paths and subsequently perform better under the original base harness.

The overall architecture comprises three core components: directional harness evolution to build a diverse pool of harness variants, complementary harness rescue training to update the shared policy, and an adaptive harness router to dynamically select the most effective variants.

Directional Harness Evolution

To construct a reusable pool of specialized variants, the authors employ a directional evolution process. Starting with a base harness $H_0$ , the system executes it on fresh training tasks to identify distinct repair directions based on local root causes of failure. An editor then modifies the parent harness components to address these specific causes, generating a set of candidate children. Parent selection for the next generation maximizes joint validation coverage, ensuring that selected parents cover each other's failures. The Sibling Harness Pool retains all valid children across generations, forming a fixed set of variants $\mathcal{H}$ that share the same execution foundation and policy parameters but differ in their context and execution rules.

Complementary Harness Rescue Training

During the training phase, the framework focuses on tasks where the current policy fails under the base harness $H_0$ . For a given task, the policy generates multiple rollouts under $H_0$ . If all attempts fail, the system triggers a rescue mechanism. Instead of discarding these failures, the policy makes fresh attempts under a set of distinct auxiliary harnesses selected from the pool. This provides the model with alternative paths through difficult tasks.

The training objective utilizes Group Relative Policy Optimization (GRPO). For a retained group of trajectories, the relative advantage is calculated as:

a_{x, j} = \frac{R_x(\tau_{x, j}) - \bar{r}_x}{\max(\sigma_x, \epsilon)}

where $\bar{r}_x$ and $\sigma_x$ represent the group's reward mean and standard deviation, and $\epsilon$ stabilizes the division. The shared optimizer forms training advantages from these group-relative rewards and maximizes a clipped token objective:

\rho = \frac{\pi_\theta(y \mid c_H)}{\pi_{\theta_{\mathrm{old}}}(y \mid c_H)}

\ell(\rho, A) = \min\{\rho A, \operatorname{clip}(\rho, 1 - \eta_-, 1 + \eta_+) A\}

where $y$ is a policy token in its generating harness context $c_H$ , and $\eta_-, \eta_+$ are the clipping widths. Only policy tokens enter the objective.

Adaptive Harness Router

To effectively select the auxiliary harnesses for rescue training, the authors design an Adaptive Harness Router. This module relies on Harness Behavior Profiles, which describe when a specific sibling harness helps or hinders the current policy relative to $H_0$ . An LLM groups behavioral differences from paired validation trajectories into structured entries detailing the task conditions and relative behaviors.

Selection is performed through a voting mechanism where multiple independent LLM voters rank the candidate harnesses based on the task context, current profiles, and validation success rates. The router aggregates these rankings to select the Top-K distinct harnesses for the rescue group. Furthermore, the system implements policy-aware profile re-anchoring, refreshing the behavioral evidence and validation statistics at specific checkpoints as the policy learns, ensuring the router's decisions remain aligned with the model's current capabilities.

Knowledge Transfer to Policy

The ultimate goal of this pipeline is to consolidate the diverse behaviors learned from sibling harnesses into the shared policy. Through the rescue updates, the policy internalizes these useful execution strategies, allowing it to achieve higher success rates even when reverting to the original base harness $H_0$ at inference time. To extract additional performance gains, the framework supports Rank-1 routing during inference, where the adaptive router selects the single best harness from the full pool, including $H_0$ , for each incoming task.

Experiment

The experiments evaluate Qwen3.5-4B and Qwen3.5-9B on AppWorld, DBBench, and SWE-Gym, comparing Harness-S with GRPO controls, fixed harness mixtures, frozen-policy harness optimization, and joint adaptation under original and task-routed harnesses. Harness-S improves task success by providing selective rescue with more within-task reward contrast and effective training tokens, while task-specific harness allocation and adaptive profile re-anchoring improve recovery and final performance. The learned policy shows changed verification and evidence-gathering behaviors that reappear on untrained instances, and routing becomes less necessary as the base harness is selected more often after training. Improvements also transfer to unseen external harnesses without further updates, though performance remains sensitive to the execution interface.

Harness-S improves long-horizon agent task success over policy-only training, frozen-policy harness optimization, and joint adaptation across AppWorld, SWE-Gym, and DBBench under both a fixed selected harness and task-specific routed harness. The routed 4B policy outperforms a larger 9B GRPO rescue baseline on every reported metric, and the routed 9B policy exceeds frontier reference models on DBBench. Gains reflect retained policy improvement plus stronger execution support, with routing further improving results without additional executor rollouts. Harness-S outperforms all compared non-frontier methods across benchmark and capacity settings under both fixed and task-specific harness selection. The routed 4B model beats a larger 9B GRPO rescue baseline on all metrics, and the routed 9B model surpasses frontier references on DBBench. Harness-S with a frozen policy outperforms Meta-Harness, showing value from sibling-pool harnesses and task-specific selection.

Across training stages, rescue success is highest with Top-8 Adaptive and its profile re-anchoring, and all rescue strategies improve final test performance over the GRPO baseline. Task-specific harness selection outperforms random selection, while broader Top-8 exploration recovers more failures than repeated sampling from fewer harnesses. Adaptive profile updating further improves routing over fixed profiles and yields the best final Pass@3 and Avg@3. Top-8 Adaptive achieves the highest rescue success at every training stage and the best final Pass@3. Profile re-anchoring improves final Avg@3 over fixed-profile Top-8, and all rescue strategies improve final Pass@3 over GRPO.

Under the base harness, Harness-S shifts policy behavior toward more schema lookups and explicit verification while reducing repeated code execution relative to GRPO. On SWE-Gym, Harness-S more often uses assertion checks and recovery behaviors when standard tests are unavailable. These patterns suggest selective rescue produces validation and interface-use behaviors that persist after auxiliary support is removed. On AppWorld, Harness-S increases schema lookup before first API use and explicit pagination while reducing repeated execute code compared with GRPO. When tests are blocked on SWE-Gym, Harness-S switches to assertion checks far more often than GRPO and more often revisits failed targets or inspects the final diff.

Successful training rescues under auxiliary support later reappear as correct held-out behavior after that support is removed. In a set-intersection case, the trained policy retains the required overlap and sends a correct email, while the baseline uses an overinclusive union and fails. In an account-recovery case, the trained policy transfers a password reset and verification procedure across applications, while the baseline fails to change the password. The trained policy applies a set-intersection rule to new song instances and sends the correct email, whereas GRPO sends an overinclusive union. A successful Venmo account-recovery rescue later transfers to an Amazon instance, including password reset and verification. GRPO requests a reset code and reads the inbox but leaves the password unchanged on the held-out account-recovery instance.

AppWorld execution scores improve across all harnesses after Harness-S training, while the router's rank ordering stays predictive. The base harness becomes a much more common top-ranked choice and surpasses the lower-ranked task-specific harnesses, though the top-ranked harness retains a modest Pass@3 edge and a clearer Avg@3 consistency advantage. Before training, all router-ranked harness choices outperform the base harness on both execution metrics, with scores declining from Rank-1 to Rank-3. After training, the base harness improves sharply and overtakes Rank-2 and Rank-3 on both Pass@3 and Avg@3, while Rank-1 still leads. The share of tasks where the router places the base harness at Rank-1 rises from 1.53% to 22.52%, with the largest preference shift occurring at Rank-1. The Rank-1 Pass@3 advantage over the base harness narrows from about 14 points to less than 1 point, and the Avg@3 advantage narrows from about 13.5 to 4.6 points. The preserved rank ordering and remaining Avg@3 margin indicate that task-specific execution support continues to improve consistency after training.

These experiments evaluate Harness-S on long-horizon agent tasks in AppWorld, SWE-Gym, and DBBench across policy-only training, frozen-policy harness optimization, joint adaptation, and task-specific routing. Harness-S consistently improves task success and execution consistency over non-frontier baselines, with adaptive Top-8 selection and profile re-anchoring yielding the strongest rescues, and routing allows a 4B policy to outperform a larger 9B GRPO rescue baseline while a 9B model surpasses frontier references on DBBench. Behavioral analyses show that successful rescues under auxiliary support persist as validation and interface-use behaviors after support is removed, and router rankings remain predictive, with task-specific harness support continuing to improve consistency after training.

Kangning Zhang Wenxiang Jiao Bowen Fang Shuai Shao Jiachen Zhu Rong Shan Shijian Wang Hao Wang Yuan Lu Weiwen Liu

Résumé

One-sentence Summary

Key Contributions

Introduction