One-sentence Summary

Researchers from TokenRhythm Technologies, Infinigence AI, Tsinghua University, Peking University, and others propose NeoHorse-1, an agent-native model family whose routing harness converts execution trajectories and capability-demand signals into training data, enabling recursive self-improvement through a three-stage curriculum and routing-guided on-policy distillation, with post-training lifting the macro-average scores of the $4\text{B}$4B and $9\text{B}$9B models from $58.94$58.94 to $64.87$64.87 and from $65.60$65.60 to $69.04$69.04 across ten benchmarks, thereby narrowing the gap between the 4B and 9B models.

Key Contributions

• Introduces NeoHorse-1, a family of agent-native models that converts routing-harness interaction records, including capability demand predictions, service tier selections, and subsequent trajectories, into user-turn training examples via structural validation, six-dimensional semantic evaluation, and subscene-level labeling.
• Organizes these examples into a three-stage supervised fine-tuning curriculum driven by routing scores, extends this staging to routing-guided on-policy distillation, and closes an evaluation-selection-update loop through capability-guided allocation of evaluation feedback into the next training mixture.
• Across ten benchmarks spanning harness-based agents, tool use, coding, and instruction following, post-training lifts the macro-average score from 58.94 to 64.87 for the 4B model and from 65.60 to 69.04 for the 9B model, with the post-trained 4B model substantially narrowing the aggregate gap to the 9B base model, providing preliminary evidence that the loop can seed subsequent iterations.

Introduction

Recursive self-improvement (RSI) in AI requires systems to observe their own capabilities and convert that evidence into subsequent learning cycles. While prior work has explored trajectory-based supervised fine-tuning, on-policy distillation, and curriculum learning, these approaches often rely on fixed teacher policies, explicit difficulty labels, or dataset-specific heuristics that are costly to obtain. Additionally, existing agentic training methods typically treat interaction records as static supervision, missing the opportunity to use routing signals that characterize capability demand and recorded outcomes that reveal model weaknesses.

The authors introduce NeoHorse-1, a family of agent-native models at 4B and 9B scales that leverage a routing-guided agentic training pipeline. Their key contribution is a closed feedback loop where a deployment harness produces three reusable signals: execution trajectories, routing-derived capability demand scores, and task outcomes. These signals organize training data into a three-stage curriculum for supervised fine-tuning and extend to routing-guided on-policy distillation, where a teacher supervises student-generated responses under the same staged progression. A capability-guided allocation step then converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop. The authors demonstrate consistent gains across ten benchmarks, lifting macro-average scores from 58.94 to 64.87 at 4B scale and from 65.60 to 69.04 at 9B scale, with the post-trained 4B model substantially narrowing the gap to the 9B base model.

Dataset

The authors build the post-training corpus around execution trajectories generated by the deployment harness, rather than static instruction-response pairs. Each trajectory captures the full interaction chain: user requests, model reasoning, tool calls, environment observations, recovery attempts, and terminal outcomes. Public data on instruction following, reasoning, tool use, code, agent interaction, and preferences supplements the harness-generated data to broaden capability coverage.

Corpus organization

• The data is structured at three granularities:
  • Trajectory: a complete interaction executed by the harness, preserving all events and outcomes.
  • User turn: begins with a user request and ends at the next request or task termination; this is the basic serialized training unit.
  • Subscene: groups adjacent user turns that share a local goal, serving as the unit for semantic characterization.
• Each user-turn record retains the current request, interleaved reasoning, tool calls, and observations, while earlier visible responses and tool interactions remain as context. Reasoning from earlier turns is omitted.
• The primary corpus contains on the order of 10^5 to 10^6 harness-generated trajectories. Scale is reported by trajectory count and token count after unified serialization, deduplication, and tokenizer freezing.

Quality control

• Deduplication removes exact and near-duplicate records. The same matching infrastructure screens all candidates against evaluation suites, removing any record that overlaps an evaluation item to keep training and evaluation data disjoint.
• Structural validation uses rule-based checks to reconstruct requests, responses, tool calls, observations, and terminal events. It verifies payload readability, message structure, causal event order, and closure of tool-call/result pairs. This stage detects missing responses, orphan observations, conflicting identifiers, unresolved calls, and ambiguous terminal branches.
• The structural gate produces three outcomes: internally complete, partially recoverable, and quarantined. Complete trajectories proceed to semantic evaluation; recoverable ones contribute only causally closed sub-trajectories; ambiguous ones are quarantined.
• Semantic evaluation assesses six dimensions: goal attainment, instruction adherence, tool use, evidence consistency, error recovery, and termination. High-certainty failures are detected deterministically, while task-level interpretation uses a semantic judge restricted to evidence in the trajectory. Each dimension gets PASS, WARN, FAIL, or NOT_EVALUATED, with coverage stored separately. Missing evidence never converts to a positive verdict.

Characterization

• Each subscene is characterized along three axes:
  • Scene: what the user is doing and in what context, using closed taxonomies for task type and application domain.
  • Goal: what the user expects to achieve and how success is judged, including acceptance criteria and cross-turn relations (new, continued, modified, resumed, ambiguous).
  • Outcome: the verifiable result of the attempt against the goal.
• Each attribute retains its derivation method and confidence. Structural facts established by the source or deterministic rules cannot be overwritten by a semantic judge.

Routing signals

• The harness router operates at the user-turn level and estimates capability demand from the current request, recent dialogue, prior routing decisions, and execution state.
• Each turn is assigned to one of four service tiers: C0 for bounded low-risk requests, C1 as the general-purpose default, C2 for multi-step reasoning and execution, and C3 for maximum capability or reliability. Policy controls may adjust the assignment.
• For each turn, the corpus retains the router's raw prediction, the policy-adjusted decision, and the tier actually served, keeping predicted demand, policy constraints, and executed action independently analyzable.

Data allocation

• At each iteration, the current model checkpoint is evaluated on a stratified suite disjoint from training. Results aggregate across attributes, quality dimensions, outcome states, and routing tiers to form a model-deficiency profile.
• This profile shifts the next training mixture toward underperforming regions while preserving broad coverage. Verified successful trajectories provide positive supervision; informative failures identify regions needing additional or rebalanced coverage.
• Between iterations, the harness continuously adds new trajectories processed by the same pipeline. Existing and new data are reallocated together, so the corpus follows changes in usage patterns and system capability.
• Routing-derived scores affect optimization through two scheduling mechanisms: they order user-turn examples in the three-stage SFT curriculum, and the same progression schedules starting contexts for on-policy distillation.

Method

The authors construct training data from execution trajectories generated by a deployment harness. A user turn is defined as a user request along with subsequent assistant responses and tool interactions. To prepare these for supervised fine-tuning (SFT), the authors serialize each turn using a specific chat template. Earlier reasoning is omitted, while visible responses and tool interactions are retained as historical context. Within the current user turn, assistant target spans (including reasoning, tool calls, and responses) receive prediction loss, whereas user messages and tool results do not. As shown in the figure below:

The SFT objective minimizes the negative log-likelihood over the masked assistant target spans. For a batch $\mathcal{B}$B of logical sequences, the loss is defined as:

where $m_{i,t}$mi,t​ is a binary token-level loss mask set to one for tokens in the retained assistant target spans.

Agentic interactions vary in capability demand, ranging from routine responses to complex planning. The authors leverage routing estimates to organize SFT examples into a progressive curriculum. Instead of relying solely on the model actually served, they re-estimate capability demand from the request and interaction history. The router assigns a tier index $k_i \in \{0, 1, 2, 3\}$ki​∈{0,1,2,3} and a normalized score vector $\pi_{i,k}$πi,k​. The authors use a soft ordering score:

This score determines the presentation order of examples across three training stages. The curriculum progressively introduces higher-scored examples while reserving some lower-scored examples for later stages to prevent the end of training from being dominated exclusively by high-demand interactions. Refer to the framework diagram:

To address the distribution shift inherent in SFT, the authors employ on-policy distillation (OPD), where a student model generates prefixes and receives teacher supervision. The routing-guided curriculum is extended to schedule the starting contexts for distillation. At each stage $j$j, a student checkpoint generates responses from contexts drawn from a distribution $\rho_j$ρj​ induced by the routing scores. The complete training process is illustrated in the figure below:

A fixed teacher supplies next-token distributions conditioned on the context and the student's preceding tokens. To compute the loss efficiently, the authors retain the student's top-$K$K candidate tokens at each position and aggregate the remaining probability mass into a single bin, creating coarsened distributions $\widetilde{P}_{\theta,r,t}$Pθ,r,t​ and $\widetilde{Q}_{r,t}$Q​r,t​. The distillation objective minimizes the response-normalized reverse KL divergence:

The stage-wise objective combines the context distribution with student generation:

The rollout checkpoint is refreshed as training proceeds, ensuring that later contexts receive teacher supervision based on more recent student behavior.

Experiment

The evaluation covers three complementary analyses: benchmark performance, agent trajectory behavior, and training data. Across agentic, coding, and instruction-following benchmarks, NeoHorse-1 improves over its base models at both 4B and 9B scales, with gains concentrated on interactive and execution-intensive tasks; the 9B variant further outperforms the 4B one, particularly in multi-step interaction and recovery from execution failures. Trajectory analysis shows that training helps close the end-to-end execution loop, while scale enables more robust iterative verification, strategy adaptation, and efficient use of interaction budgets under feedback and failure. Data analysis reveals that routing-harness trajectories transfer more effectively than public synthetic agent data under matched training conditions, and that scaling high-quality routing-harness supervision yields consistent aggregate performance gains across the evaluated range.

NeoHorse-1-4B demonstrates competitive or superior performance against representative 4B-scale open-weight models across agentic, coding, and instruction-following benchmarks. It achieves the highest average score among the compared models, with notable strengths in multi-turn tool use and instruction following, while trailing in some coding-specific tasks. NeoHorse-1-4B attains the best average score across all benchmarks compared to other 4B-scale models. It leads in instruction-following metrics, with top scores on IF Bench and IF Eval. Its agentic performance is strong, particularly on VitaBench and PinchBench, where it outperforms most peers. Coding results are mixed: it ranks second on HumanEval but lower on LiveCodeBench v6 relative to some competitors.

NeoHorse-1-9B outperforms larger models on most agentic, coding, and instruction-following benchmarks, achieving the highest average score among the compared models. It excels particularly in agentic tasks and instruction following, while showing competitive coding performance despite its smaller size. NeoHorse-1-9B achieves the best average score across all benchmark categories, surpassing larger models like Muse-Glimmer-30B. It leads in agentic benchmarks, including top scores on BFCL v4, VitaBench, and WorkBuddyBench, and ties for best on PinchBench. On instruction following, it records the highest IFBench score and a strong IFEval result, outperforming all larger models. In coding, it trails only the larger Muse-Glimmer-30B on HumanEval and LiveCodeBench, while still exceeding several other larger models.

Routing-harness trajectories outperform public tool-agent data under the same training configuration, yielding consistent gains across all five benchmarks. The largest improvements are seen in code generation and multi-turn tool use, while smaller but positive gains appear in function calling and instruction following. Routing-harness data improves the average benchmark score by 6.26 points over public agent data. The biggest gains are in HumanEval (+8.54) and τ²-Bench (+11.31), indicating stronger coding and multi-turn tool-use supervision. Function calling, instruction following, and coding benchmarks also improve, showing broad transferability of routing-harness trajectories.

NeoHorse-1-4B matches or exceeds other 4B-scale models on agentic, coding, and instruction-following benchmarks, achieving the top average score with particular strengths in multi-turn tool use and instruction following, though it lags in some coding tasks. NeoHorse-1-9B outperforms larger models on most benchmarks, leading in agentic tasks and instruction following while remaining competitive in coding despite its smaller size. Additionally, routing-harness trajectories consistently beat public tool-agent data under identical training, with the largest gains in code generation and multi-turn tool use, improving the average score by 6.26 points.