One-sentence Summary

DarwinX, introduced by researchers at Salesforce AI Research and Salesforce Agentforce, evolves populations of agent harnesses through natural selection with the model frozen, using preserve-and-extend edits, lineage recombination, and shared evidence interfaces to avoid path-dependent regression, and across four benchmarks it adds roughly 17 points on average and transfers unchanged to SWE-bench Verified.

Key Contributions

• DarwinX improves a frozen-model agent by treating self-evolution as selection over a population of harness variants, combining a preserve-and-extend contract, an archive of alternative lineages, and a shared edit interface for failure-, teacher-, and self-derived evidence.
• Fitness comes from each benchmark's own verifier with no gold solutions or hand-picked winners, yielding an average gain of about 17 points across four benchmarks; Terminal-Bench 2.1 rises from 75.5% to 83.2% on GPT-5.5 and to 84.7% on GPT-5.6 Sol, TerminalWorld's held-out split reaches 68.3%, and WebArena-Infinity audit-clean pass@1 rises from 43.5% to 93.0%.
• A Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified at 84.2%, and the gains reflect general agent competence rather than benchmark-specific patches, so they survive changes of task, verifier, and base model.

Introduction

The authors study self-improving LLM agents, where capability depends on the harness (prompts, tools, memory, and control flow) as well as the frozen model. Prior self-editing systems share an inner loop of batch rollouts, reflection, bounded edits, and regression gating, but single-lineage search suffers from path dependence and plateaus, and edits that fix one task family often regress another, especially on mixed benchmarks. Existing archive-based approaches can preserve variants but tend to leave complementary specialists isolated and do not require new gains to retain prior solutions. The authors propose DarwinX, which recasts self-evolution as population selection over harness variants, adds a preserve-and-extend contract to bound regression risk, and archives variants so complementary specialists can be inherited and recombined. Selection is driven by measured benchmark fitness without gold solutions or hand-picked winners.

Dataset

The authors use two disjoint data components for synthetic-to-real generalization:

• Synthetic evolution set: 300 synthetic intents generated from each application’s own description document. The generation pipeline never reads the benchmark’s task suites, so the synthetic set is kept separate from real evaluation tasks. An LLM judge scores synthetic trajectories using avg@3 for screening and avg@5 for confirmation. No real WAI task or verifier influences selection.

• Real held-out evaluation set: 1,260 real tasks, disjoint from the synthetic intents. These tasks are accompanied by deterministic verifiers and are not visible to the evolution loop at any point.

Processing and usage: The synthetic intents are used to generate trajectories that drive iterative selection during evolution. The real tasks are reserved only for final reporting, where pass@1 is computed. On the synthetic full set, two intermediate checkpoints score 38.4% and 34.4%, compared with 19.7% for the base model. The selection gate keeps 26 iterations and reverts 36, so accepted improvements accumulate along a short primary lineage rather than through merged branches.

Method

The authors leverage a selection-based approach to agent improvement called DarwinX, which treats capability gains as an evolutionary problem rather than a training problem. The base model weights remain entirely frozen throughout the process. Instead, the system iteratively proposes small edits to the agent harness, comprising prompts, memory, tools, and control flow, and retains an edit only when empirical evidence demonstrates it solves new tasks without regressing on previously solved ones.

The core of this framework is the preserve-and-extend contract. A child harness $c$c is evaluated against its parent $p$p using per-task solve rates. The net gain is defined as $g(c) = \sum_t \Delta_t$g(c)=∑t​Δt​ where $\Delta_t = \hat{p}_t(c) - \hat{p}_t(p)$Δt​=p^​t​(c)−p^​t​(p), and the bounded regression is $R(c) = \sum_t (-\Delta_t)_+$R(c)=∑t​(−Δt​)+​. A fitness enabler admits a child if $g(c) > 0$g(c)>0 and $R(c) \le \delta$R(c)≤δ. A reasoned verifier agent $f$f then adjudicates in two stages: an initial promotion based on trial evidence $\mathcal{E}$E and shared memory $K_g$Kg​, followed by a stricter preservation probe. This two-speed design allows rapid exploration while preventing lucky rollouts from corrupting the search.

Branch evolution maintains an archive shaped as a tree. Parent selection ranks nodes by cumulative lineage gain $G(c) = G(p) + g(c)$G(c)=G(p)+g(c). The next parent is sampled as $p^* \sim (1-\beta)\delta_{\text{argmax}_{v \in S}} G(v) + \beta \text{Broaden}(P)$p∗∼(1−β)δargmaxv∈S​​G(v)+βBroaden(P), balancing exploitation of the highest-gain confirmed variants with broadening across the wider population.

Population-level inheritance and recombination ensure that capabilities accumulate rather than trade off. Variants are classified by how their solved task set $S(c)$S(c) compares to the parent $S(p)$S(p). Improvers ($S(c) \supseteq S(p)$S(c)⊇S(p)) and neutral children ($S(c) = S(p)$S(c)=S(p)) remain eligible for recombination, while stepping stones and archived variants contribute only distilled lessons. When complementary variants $v_1, \dots, v_n$v1​,…,vn​ solve different tasks, DarwinX materializes an inherited child by merging their additive edits from a common ancestor $H_0$H0​. The merged harness is $H = H_0 \oplus \Delta$H=H0​⊕Δ, where $\Delta = \Delta_{\text{code}} \oplus \Delta_{\text{skill}} \oplus \Delta_{\text{prompt}} \oplus \Delta_{\text{tool}}$Δ=Δcode​⊕Δskill​⊕Δprompt​⊕Δtool​. The child is retained only if it covers the union of its parents wins, $S(\text{child}) \supseteq \bigcup_i S(v_i)$S(child)⊇⋃i​S(vi​).

The mutation loop is driven by a modular learning-signal interface. Before proposing an edit, the system selects the most informative evidence available. Failure-derived signals ($\nabla$∇) summarize failed trajectories to localize missing capabilities. Teacher-derived signals ($\pi^*$π∗) distill successful reference trajectories for tasks where the agent has no successful rollout. Self-derived signals ($A$A) contrast the agent own passing and failing rollouts to identify reliable success factors. All signals are translated into candidate harness edits without updating model weights.

Measurement and confirmation rely on binary avg@k evaluation. Agent timeouts count as failures, and a preservation probe re-samples the lineage known solved set before a promoted child becomes an ancestor. Finally, a failure-mode classifier aggregates dominant themes across the benchmark into a population-wide shared memory $K_g$Kg​, updated as $K_{g+1} = \text{Agg}(K_g, \text{worked}, \text{regressed}, \text{themes})$Kg+1​=Agg(Kg​,worked,regressed,themes). This memory is read by both the proposer and the verifier, enabling the search to invent global capabilities that address systemic bottlenecks rather than isolated per-task patches.

Experiment

DarwinX is evaluated as a general harness-optimization pipeline across five research questions: four benchmarks with increasing separation between evolution signal and test, plus an ablation. On frozen models, the evolved harness consistently improves over base Monet, with gains concentrated in procedurally difficult tasks and attributed to verification and artifact-contract skills rather than added compute or verifier gaming. Held-out and synthetic-to-real experiments show generalization beyond training tasks, while cross-benchmark transfer to SWE-bench Verified yields a smaller improvement and the ablation confirms preserve-and-extend behavior without cluster-level regression.

Self-improving-agent methods share the same inner loop but differ in the search and selection mechanisms wrapped around it. Existing optimizers and scaffold editors largely lack cross-lineage merging and bounded regression, leaving them exposed to path dependence and cross-task interference. The distinguishing approach combines a population archive, a preserve-and-extend contract, and noise-aware selection so complementary specialists can be recombined without silently regressing other tasks. Compared methods mostly omit bounded regression and cross-lineage merge, with population archive use restricted or absent in several prior optimizers. Genetic prompt optimizers recombine prompt strings, but only in a restricted way that does not merge variants selected for solving complementary tasks.

An evolved harness on a frozen model reaches the frontier of the Terminal-Bench verified leaderboard. It edges past the prior verified leader at a lower effort setting and adds several points over the base harness and a neutral high-effort harness. The improvement is attributed to targeted test-time compute rather than a larger model or a uniformly higher budget. The frozen-base evolved harness leads the verified leaderboard, slightly ahead of the prior leader despite using a lower effort setting. On a matched base model, the evolved harness improves from 75.5% to 83.2% and beats a neutral high-effort harness by more than five points. Extra compute is concentrated on tasks that newly pass, while already solved tasks consume nearly the same resources.

On TerminalWorld's held-out split, the evolved Monet harness paired with Opus 4.8 achieves the highest single-attempt pass@1 among evaluated agents, ahead of Claude Code and the unevolved base harness. Matched-model comparisons show a similar absolute improvement from the evolved harness on both Opus 4.8 and GPT-5.5. The split is disjoint from training tasks and frozen before evaluation, so gains reflect held-out generalization rather than retries or training-task replay. The evolved Monet harness leads the held-out pass@1 ranking, outperforming frontier coding agents, neutral terminal agents, and its unevolved base. Evolution adds a comparable number of solved held-out tasks across two frozen base models. Single-attempt evaluation on unseen tasks indicates that the improvement reflects generalization rather than training-task replay or retry effects.

Monet with DarwinX achieves the best audit-clean pass@1 on the real WebArena-Infinity suite, reaching 93.0% overall and outperforming the strongest same-model baseline and top public agent by clear margins. Relative to base Monet on the same frozen model, the evolved harness more than doubles the audit-clean pass rate and shows gains across every reported application, with the largest increases on state-change-heavy workflows such as prescriptions and Gmail. Monet (DarwinX) reaches the highest overall audit-clean pass rate, surpassing the same-model GPT-5.5 + Browser Use baseline and the top public agent by meaningful margins. The gain over base Monet is broad rather than concentrated: every reported application improves, and state-change-heavy tasks such as prescriptions and Gmail show the largest increases.

The evolved harness improves both raw and audit-clean pass rates over base Monet, with audit-clean pass@1 rising from 43.5% to 93.0%. Invalid successes drop from 120 to 17, and confirmed invalid, human review, and blocked attempt rates fall to near zero. Evaluation-plane, privileged-host, and exploit-related violations disappear, leaving only raw-state mutations. Audit-clean pass@1 rises from 43.5% to 93.0%, while confirmed invalid successes fall by 22.1 percentage points. The evolved harness eliminates evaluation-plane, privileged-host, and exploit violations entirely; the remaining invalid trajectories are all raw-state mutations.

The experiments evaluate an evolved agent harness on frozen models across Terminal-Bench, a held-out TerminalWorld split, and the real WebArena-Infinity suite. The evolved harness reaches or surpasses prior leaderboard and frontier agents even at matched or lower effort, and gains transfer to unseen tasks, indicating held-out generalization rather than retry or replay effects. On the real web benchmark, improvements are broad across applications and the harness nearly eliminates invalid, privileged-host, and exploit-related trajectories. Overall, the results validate that evolving scaffold designs with bounded regression and cross-lineage merging yields more robust, transferable, and audit-safe agent behavior.