One-sentence Summary

Researchers from Westlake University and Ant International propose Trajectory2Gym, which constructs executable tool-use interactions around under-supported compositional structures and supervises only decision-grounded actions using structural-support selection and parent-normalized fine-tuning, improving Tau1, Tau2, and BFCL by an equal-weight three-benchmark mean of 16.9 to 29.8 percentage points across five Qwen3 scales.

Key Contributions

• The paper introduces Trajectory2Gym, a trajectory construction framework that builds executable tool-use interactions around under-supported compositional structures and derives supervision only from decisions grounded in the learner-visible interaction history.
• Trajectory2Gym contributes structural-support selection for allocating the resulting experience and parent-normalized supervised fine-tuning that keeps the complete semantic interaction as the unit of training weight.
• Experiments across five Qwen3 scales show that standard supervised fine-tuning on the constructed interactions improves Tau1, Tau2, and BFCL at every tested scale, increasing the equal-weight three-benchmark mean by 16.9 to 29.8 percentage points, with the largest gains on stateful and multi-turn tasks.

Introduction

Tool-use agents must learn compositional interactions, where observations, user updates, or state changes determine later decisions in multi-turn workflows such as customer support and API use. Prior training pipelines often filter trajectories by task completion or per-call execution, but success alone does not guarantee that each imitated action was justified by the agent-visible history, and logged data often lacks sufficient coverage of cross-step dependencies and recovery after authentic errors. The authors introduce Trajectory2Gym, which synthesizes under-supported interaction paths, certifies supervision only for decisions grounded in visible evidence and workflow order, and applies structural-support selection with parent-normalized supervised fine-tuning.

Dataset

The authors construct a synthetic interaction dataset from compositional capability paths executed in native environments.

• Sources and composition: A shared capability vocabulary covers semantic actions and arguments, observation outcomes, recovery, branching, handoff, termination, and horizon. Native tools and observations map to these roles. An ordered path composes required capabilities into one interaction. A generator creates a request and compatible initial state for a target path, and a teacher executes the task in the native environment. Each trajectory is assigned one canonical path for support accounting.

• Certification and filtering: A path is certified only when its action and observation dependencies appear in the executed log. Only certified structures update the support inventory and guide later synthesis rounds. The method targets paths whose support falls below the scale-specific target and conditions the generator on them. Construction establishes entities, prerequisites, and information sources for each path, and binds required values to visible sources such as the request, a user exchange, or a tool result.

• Dataset assembly: The authors assemble $\mathcal{D}_s$Ds​ from the certified pool. Selection preserves task breadth, covers capability paths, and gives repeated support to under-represented structures such as discovery, recovery, and completion. The target number of distinct trajectories per feasible canonical path grows with learner scale: 1, 2, 3, 4, and 8 from 1.7B to 30B-A3B. Selection starts from a fixed certified core, adds one completing exemplar per training task, closes uncovered atoms and paths, and then adds trajectories that reduce support deficits.

• Processing and schema: Selected trajectories are rendered through native prompts, roles, tool schemas, and visible observations. One semantic trajectory may yield several training examples. Non-target visible messages remain context.

• Use in the model: For supervised fine-tuning, the authors use parent-normalized SFT. Each parent trajectory receives equal weight, and its weight is distributed across rendered examples in proportion to their certified target-token counts. This prevents long multi-turn interactions from dominating per-example averaging. The provided excerpt does not describe an explicit train/validation/test split or mixture ratios beyond the scale-dependent per-path support targets.

Method

The authors propose Trajectory2Gym, a framework designed to construct high-quality supervised trajectory sets for tool-use agents by addressing compositional coverage and decision grounding. The overall pipeline consists of three tightly coupled stages: compositional interaction synthesis, grounded supervision, and structural-support learning.

Refer to the framework diagram:

In the first stage, compositional interaction synthesis, the authors turn under-supported interaction paths into executed native interactions. A shared capability vocabulary covers semantic actions, arguments, observation outcomes, recovery, branching, and termination. A generator constructs a request and a compatible initial state for a target path, and a teacher executes the task in the native environment. Each trajectory is assigned a canonical path for support accounting. Native execution supplies authentic observations, state changes, and rejections, ensuring that teacher-only guidance stays out of the student trace. A path is certified only when its action and observation dependencies appear in the executed log.

The second stage, grounded supervision, transforms each executed log into a student context and a set of certified targets. This stage ensures that each supervised action is supported by the realized interaction and its decision-time context.

As shown in the figure below:

The authors distinguish native execution validity $e_{t} \in \{0, 1\}$et​∈{0,1} from decision-time grounding $v_{t} \in \{0, 1\}$vt​∈{0,1}, defining local eligibility as:

The decision context contains only events exposed through the native agent interface; teacher hints and private events never enter the student context. For a tool call, execution validity requires authenticated non-error execution, while decision-time grounding requires the tool to be available and identifier-like arguments to be supported by preceding visible information. The final supervision mask incorporates native terminal success and ordered subsequence inclusion of the task requirements:

where $R_{\tau}$Rτ​ is native terminal success, $q_{\tau}$qτ​ is the task ordered requirements, and $A_{\tau}$Aτ​ is the canonicalized sequence of relevant operations. Rejected calls and their authentic errors remain visible context without imitation loss, while the grounded recovery that follows becomes a target.

In the third stage, structural-support learning, the authors assemble the final dataset to preserve task breadth and cover capability paths, giving repeated support to under-represented structures such as discovery, recovery, and completion. For a given learner scale, a target number of distinct trajectories per feasible canonical path is defined. The remaining deficit of a selected set is calculated as:

The selection process starts from a fixed certified core, adds completing exemplars per training task, closes uncovered atoms and paths, and then adds trajectories that reduce the support deficits.

For training, the authors employ parent-normalized supervised fine-tuning (SFT). Since one semantic trajectory may yield several training examples, per-example averaging would cause long multi-turn interactions to dominate the loss. Instead, each trajectory contribution is defined at the parent level. The loss for a trajectory is calculated as the average negative log-likelihood of its certified next-token pairs:

This approach ensures each parent receives equal weight regardless of rendering multiplicity, keeping the semantic training mass independent of how many examples a trajectory is rendered into.

Experiment

Trajectory2Gym is evaluated by fine-tuning Qwen3 models from 1.7B to 30B on scale-conditioned trajectory data and testing them on Tau1, Tau2, and BFCL, with paired comparisons against each model's own initialization and against public agent-training checkpoints. The experiments show that standard SFT on decision-grounded compositional data improves stateful tool use at every tested scale, with gains concentrated in multi-step and multi-turn tasks, and smaller post-trained models can outperform much larger RL-trained checkpoints. Matched ablations indicate that targeted expansion, grounding, structural selection, and parent normalization all contribute, with parent normalization giving the largest isolated gain, while sensitivity analysis shows that deeper structural support is not automatically better. Qualitative cases confirm that the method supervises evidence-grounded decisions and order-sensitive workflow recovery, behaviors that baseline agents fail to exhibit.

Trajectory2Gym improves stateful tool-use results on Tau1 and Tau2 at every tested model scale, with mean gains between 20 and 41 points and broad paired improvements. Post-trained models outperform larger public initializations and re-evaluated agent-training checkpoints, and the largest post-trained model approaches closed-source reference scores. Strong function-calling results alone do not transfer to stateful tasks. Tau1 and Tau2 mean scores improve by 20 to 41 points at every tested scale, and 24 of 25 paired domain-scale comparisons improve. The paired gain is smallest at 1.7B scale; from 4B upward it is 35 to 41 points. A post-trained 4B model reaches 69.17 on Tau2, far above a public 30B-A3B initialization, and a post-trained 30B-A3B model comes within 1.2 points of closed-source reference scores. An 8B Trajectory2Gym model reaches 74.17 on Tau2, at least 28 points above all re-evaluated agent-training checkpoints, including 14B and 32B models. Function-calling skill does not transfer to stateful tasks: an EnvFactory-8B model nearly matches the 8B model on BFCL Overall but trails by 47.5 points on Tau2.

In this BFCL comparison, the reported closed-source models lead overall, with Gemini-3.0-Pro scoring highest and GPT-5.5 next. ToolACE-2-8B is competitive overall but has a much lower multi-turn base score, while Simia Tau-SFT trails by a wide margin because its parallel non-live and multi-turn base scores are near zero. Gemini-3.0-Pro leads overall, followed by GPT-5.5, with both closed-source models ahead of the listed agent-training references. ToolACE-2-8B performs strongly in non-live parallel, multiple, and irrelevance categories but drops sharply in multi-turn base. Simia Tau-SFT has near-zero parallel non-live and multi-turn base results, producing the lowest overall among the reported models.

All four matched component comparisons favor the proposed Trajectory2Gym choices over stage-matched alternatives, with positive mean gains. The gains are directional and not additive because each pair changes a different stage. Parent normalization provides the largest isolated improvement, supporting semantic interaction as the unit of optimization. All four matched comparisons yield positive mean gains, with parent normalization showing the largest isolated gain. Targeted expansion mainly strengthens stateful Tau tasks, decision grounding improves decision-level supervision, and structural selection covers more canonical paths at the same budget.

In this sensitivity run, benchmark mean declines as structural-support depth increases, with the shallowest setting performing best despite having the fewest trajectories and target tokens. The drop is concentrated in Tau2 while BFCL stays relatively stable, and because depth also changes supervision volume the result does not isolate path support from token count. Mean benchmark performance is highest at the shallowest support depth and decreases as depth increases. Tau2 shows the largest decline, while BFCL remains comparatively stable across support depths.

The experiments evaluate Trajectory2Gym on the stateful tool-use benchmarks Tau1 and Tau2, compare function-calling and BFCL behavior, ablate the proposed components, and test structural-support depth. Trajectory2Gym improves stateful tool-use across all model scales and outperforms larger public initializations and agent-training checkpoints, while function-calling skill alone does not transfer to stateful tasks. Closed-source models still lead BFCL overall, and reference agent-training models show notable weaknesses in multi-turn or parallel non-live settings. Ablations show all proposed components help, with parent normalization providing the largest isolated gain, and sensitivity results favor shallow structural support, though depth is confounded with supervision volume.