One-sentence Summary

Researchers from Alibaba Group’s DreamX Team present LongHorizon-Harness, which treats long-horizon agent execution as explicit task-state management using a Manage-Execute-Audit loop with fresh-context executors, read-only auditors, and a pluggable AgentAdapter, improving Qwen 3.7-Plus from $51.8\%$51.8% to $80.7\%$80.7% on WeaveBench, $69.7\%$69.7% to $77.2\%$77.2% on Terminal-Bench 2.1, and $2.8\%$2.8% to $8.3\%$8.3% on OSWorld 2.0, while raising Claude Opus 4.7 from $20.0\%$20.0% to $34.3\%$34.3% on an OSWorld 2.0 subset.

Key Contributions

• Long-horizon agent execution is reformulated as a task-state management problem, maintaining progress in an explicit state record outside task execution and updating it only with facts independently verified from the environment.
• LongHorizon-Harness implements a Manage-Execute-Audit loop where a manager defines one subtask from the current state, an executor completes it in a fresh context, and a read-only auditor independently inspects the environment before the manager updates the state for the next round.
• On WeaveBench, OSWorld 2.0, and Terminal-Bench 2.1, LongHorizon-Harness raises Qwen 3.7-Plus scores from 51.8% to 80.7%, 2.8% to 8.3%, and 69.7% to 77.2% respectively, and improves Claude Opus 4.7 from 20.0% to 34.3% on an OSWorld 2.0 subset.

Introduction

The authors address the growing need to delegate complex, multi-step work to autonomous agents based on large language models, where success hinges on sustaining coherent progress over long horizons. Prior agent harnesses help manage planning and tool use but suffer from two structural issues: the execution history and task state share one expanding context, leading to difficulty in tracking progress, and subtask execution remains coupled with completion assessment, so erroneous judgments can persist. They contribute LongHorizon-Harness, a framework that reframes long-horizon execution as a task-state management problem. It maintains an explicit, independently audited task state outside execution, updating it only with environment-verified facts, and organizes work into a Manage-Execute-Audit loop where each subtask runs in a fresh context, discarding raw interaction history and carrying forward only compact, confirmed state. This decoupling delivers large improvements on WeaveBench, OSWorld 2.0, and Terminal-Bench 2.1, nearly doubling prior best results in some settings.

Method

The authors propose LongHorizon-Harness, a framework designed to execute long-horizon tasks in a computer environment through a sequence of dynamically determined rounds rather than a single continuously growing session. The system maintains an explicit task state outside of task execution and advances it only with evidence independently verified from the environment. Across rounds, only the task state and its supporting audit reports persist, while the raw interaction trajectory of the executor is discarded after each round.

Each round follows a Manage-Execute-Audit (MEA) loop. Let $S_i$Si​ denote the task state available at the beginning of round $i$i, $e_{i-1} \in \mathcal{E}$ei−1​∈E the current environment state, and $V_{i-1} = (v_1, \dots, v_{i-1})$Vi−1​=(v1​,…,vi−1​) the accumulated audit reports. The manager constructs a bounded subtask contract $c_i$ci​. A fresh-context executor performs the contract, transforming the environment from $e_{i-1}$ei−1​ to $e_i$ei​, and returns an execution report $o_i$oi​. A separate auditor then inspects $e_i$ei​ through read-only tools and produces audit report $v_i$vi​. The manager incorporates $v_i$vi​ into the next task state $S_{i+1}$Si+1​ before determining whether another round is required. The loop terminates when the audited state satisfies the original task, no permitted subtask can advance the remaining requirements, user input is required, or the round budget is exhausted.

As shown in the figure below:

The Manager owns the persistent task state and determines how the task should proceed. It has access to the original task $\tau$τ, the current task state, and all accumulated audit reports, but has no direct interface to the computer environment. Its decisions are based entirely on the task state and environment evidence recorded by the auditors. After round $i$i, the manager updates the task state and produces the next control decision:

where $V_i = (v_1, \ldots, v_i)$Vi​=(v1​,…,vi​) and $q_{i+1}$qi+1​ is one of execute, done, blocked, and ask. The contract $c_{i+1}$ci+1​ is returned only when further execution is required. The task state is a structured collection of task-relevant records, including requirements, artifacts, and facts. Each record is marked as completed, pending, blocked, or untrusted, and retains references to the audit evidence supporting its current status. Executor claims do not directly change the persistent state; a record is marked as completed only when supported by clean audit evidence.

The Executor performs the contract selected by the manager and is the only role permitted to intentionally modify the environment. In round $i$i, it receives the original task $\tau$τ, the current task state $S_i$Si​, the subtask contract $c_i$ci​, and only the prior audit reports referenced by the contract. It transforms the environment from $e_{i-1}$ei−1​ to $e_i$ei​:

where $o_i$oi​ summarizes the actions performed, resulting state, artifacts produced or modified, and issues encountered. Each executor invocation runs as a fresh, budget-bounded episode containing only the information supplied for the current round. GUI and CLI executors operate through different environment interfaces. The GUI executor handles screen-oriented capabilities, while the CLI executor handles shell execution, file editing, coding, and testing. Executors are instantiated through a common agent-adapter interface, launching existing backends as bounded episodes.

The Auditor independently verifies the environment state produced by the executor. It receives the original task $\tau$τ, the task state $S_i$Si​, the contract $c_i$ci​, the prior audit reports referenced by the contract, and the executor report $o_i$oi​. It inspects the resulting environment $e_i$ei​ and produces:

where $v_i$vi​ is appended to the persistent audit history and returned to the manager. The auditor starts from a fresh context that excludes the raw interaction trajectory of the executor. It determines completion by independently comparing the resulting environment against the goal, acceptance criteria, and boundary constraints in $c_i$ci​. The auditor possesses read-only authority and cannot modify task-relevant environment state. The audit report records completion status, integrity status, and task-state updates supported by the inspection.

Experiment

The evaluation spans WeaveBench, OSWorld 2.0, and Terminal-Bench 2.1, comparing LongHorizon‑Harness against a matched Claude Code baseline with the same Qwen and Claude Opus backbones. Explicit task‑state management and independent auditing improve success rates across all three environments, with manager overhead staying under 10% of tokens while total cost can decrease for stronger models. Gains are largest on tasks requiring sustained verification and recovery of multiple dependent states, and case studies confirm that persisting audited progress outside execution history enables fresh‑context executors to convert near‑failure trajectories into complete results.

Using Qwen 3.7-Plus with Claude Code as the executor, LongHorizon-Harness raises the full-task pass rate on WeaveBench from 51.8% to 80.7% and the mean score from 0.702 to 0.835. The gain appears across all eight task domains, showing that explicit task-state management delivers broad improvements rather than being limited to specific categories. LongHorizon-Harness improves the pass rate from 51.8% to 80.7% in a matched comparison with the same model and executor backend. The mean task score increases from 0.702 to 0.835 when adding the task-state management layer. Performance rises across all eight benchmark domains, confirming that the benefit is not concentrated in a single application area.

LongHorizon-Harness with Qwen 3.7-Plus triples binary completion on OSWorld 2.0 (from 2.8% to 8.3%) and lifts the partial score from 21.5% to 35.2%, demonstrating that explicit task-state management substantially improves long-horizon computer use. Official results show that even the strongest model, Claude Opus 4.8, reaches only 20.6% binary completion, highlighting the difficulty of the benchmark. LongHorizon-Harness with Qwen 3.7-Plus increased binary completion threefold, from 2.8% to 8.3%. The partial score rose from 21.5% to 35.2%, indicating that agents fulfilled a larger share of task requirements on average. The best official configuration, Claude Opus 4.8 with batched actions, achieved 20.6% binary completion, underscoring the challenge of OSWorld 2.0. Batched-action Claude Opus 4.8 outperforms its single-action counterpart (20.6% vs 18.5% binary), yet both remain well below full task completion.

On a 34-task OSWorld 2.0 subset, LongHorizon-Harness boosts Claude Opus 4.7's binary completion from 20.6% to 35.3% and its partial score from 55.8% to 66.9%. The improvements mirror those achieved with a different backbone, showing that explicit task-state management adds complementary value beyond stronger model actions. Binary completion rises sharply from 20.6% to 35.3%, meaning the agent fully completes the workflow more often. Partial score increases by over 11 percentage points, confirming more task requirements are satisfied, and the gains are consistent across different backbone models.

On the WeaveBench Games subset, LongHorizon-Harness transforms near-zero scores from Qwen 3.7-Plus into moderate successes, but at a steep token cost. For the stronger Claude Opus 4.7, the harness produces small score changes while consistently slashing token consumption. This shows that the harness’s cost-performance trade-off depends heavily on the executor model’s capability. For Qwen 3.7-Plus, LongHorizon-Harness improved all five task scores, with the largest gains on tasks where the baseline model scored zero (stockfish_puzzle, mines_visual) or near-zero (mines_solve). Qwen’s token usage surged under the harness, increasing from 0.9M–10.0M to 13.9M–97.2M per task, while Claude’s token usage dropped in every task. Claude Opus 4.7 experienced score regressions on mines_solve and stockfish_puzzle, yet token usage declined by 44–85% across all tasks. On game_ui_bug, both models gained from the harness (Claude +0.08, Qwen +0.10), but Claude used 29% fewer tokens and Qwen used 3.4× more.

Evaluation on WeaveBench and OSWorld 2.0 shows that LongHorizon-Harness substantially raises task completion rates and mean scores, with improvements observed across all eight task domains and when paired with different base models. On the WeaveBench Games subset, the harness dramatically lifts a weaker model's scores at a large token cost, whereas it mainly reduces token usage for a stronger model with mixed score effects, indicating that explicit task-state management shifts from enabling completion toward enhancing efficiency as model capability grows.