One-sentence Summary

ByteDance and the University of Science and Technology of China propose DART-SD, a diamond-topology aware retrieval and tuning framework that replaces full-trajectory imitation with topology-guided localized correction via an Interaction-State Transition Graph, Critical Topological Breakpoint retrieval, and progressive self-distillation, thereby improving policy diversity and outperforming full-trajectory baselines on complex multi-turn tool-calling benchmarks.

Key Contributions

• An Interaction-State Transition Graph (ISTG) models the execution process of multi-turn tool-calling tasks, capturing the diamond topology formed by order-independent valid explorations.
• A localized correction method identifies the Critical Topological Breakpoint (CTB) in a failed trajectory, retrieves success-supported recovery references from the ISTG, and applies supervised loss only on the generated recovery steps while preserving the valid reasoning prefix.
• A progressive self-distillation paradigm repeatedly rolls out the student model, identifies its evolving capability boundary, and applies CTB-guided localized supervision to iteratively extend valid interaction prefixes and gradually master more complex tool-use behaviors.

Introduction

Equipping large language models with multi-turn tool-calling capabilities is central to building autonomous agents, but state-of-the-art performance relies on expensive frontier models, which motivates distilling their planning and interaction skills into compact open-source models. Prior distillation and reinforcement learning methods often treat multi-turn tool use as isolated linear trajectories. Behavior cloning and supervised fine-tuning can overwrite valid exploratory steps through indiscriminate global loss, hindsight approaches enforce rigid linear sequences that confuse fatal errors with harmless exploration, and RL methods such as GRPO suffer from credit misassignment by penalizing valid steps inside failed trajectories. Because order-independent subgoals cause distinct trajectories to intersect in a diamond-shaped solution space, flattening this structure into linear paths produces a topological collapse. The authors propose DART-SD, which builds an Interaction-State Transition Graph over cumulative interaction states, identifies Critical Topological Breakpoints by projecting student states onto the success-reachable region, retrieves success-supported recovery continuations, and applies progressive self-distillation with localized supervision. This approach updates only recovery steps, preserving valid reasoning prefixes and improving token efficiency.

Dataset

The authors construct a task-specific interaction-state graph dataset from tool-use rollouts. The excerpt does not state the raw corpus size or source name; it describes how teacher rollouts are processed into an ISTG.

Sources and composition

• Data consist of teacher tool-use rollouts per task, including both successful and failed rollouts.
• Each rollout is a sequence of tool calls or concurrent call bundles and their responses.
• All rollouts share a common root state and terminate in either a success or failure terminal.
• For each task $x$x, useful response content is abstracted into a task-specific set of information atoms $\kappa_x$κx​.

Processing and filtering

• Deterministic canonicalization: Each tool response is parsed into fields. A response is non-informative only if every field is a status signal, empty, or placeholder. Error and not-found messages become non-informative; a response with at least one substantive payload field remains informative.
• Non-informative responses are collapsed per tool into a single class, which removes much of the corpus without semantic judgment.
• Semantic atomization: Surviving responses for a task are judged jointly, conditioned on the task question and one successful rollout. Each pair of tool identity and canonicalized response maps to at most one atom.
• Informative responses receive a singleton atom; non-informative or unjudged responses map to empty.
• Equivalent facts receive the same atom even when produced by different tools or in different surface forms, so acquisition paths can reconverge.

State and graph construction

• The interaction state at step $t$t is $X_t=(I_t,U_t)$Xt​=(It​,Ut​), where $I_t$It​ is the set of acquired information atoms and $U_t$Ut​ is the multiset of useless operations since the most recent main node.
• The root state is $(\varnothing,\varnothing)$(∅,∅) and is a main node. Later states are main nodes when the step adds previously unseen atoms, and auxiliary nodes when no new atom is added and useless operations are recorded.
• For each task, a directed multigraph $G_x=(V_x,E_x)$Gx​=(Vx​,Ex​) is built. Vertices represent main or auxiliary states; edges represent one tool call or concurrent call bundle. Parallel edges are retained.
• Main nodes form the information-acquisition backbone, while auxiliary nodes represent useless exploration attached to the most recent main node.
• No explicit dataset size, split ratios, cropping strategy, or external metadata construction is given in this excerpt.

How the data is used

• The graphs serve as shared interaction-state spaces.
• Student rollouts are replayed under the same state-update rule:

• Teacher and student executions are embedded in the same state space, enabling separate matching of information acquisition and useless exploration without requiring action-wise correspondence.

Method

The authors propose the DART-SD framework to enhance tool-use capabilities through topology-aware recovery and progressive self-distillation. The overall pipeline first constructs an Interaction-State Transition Graph (ISTG) from teacher rollouts, identifies the Critical Topological Breakpoint (CTB) of failed student rollouts via success-reachable projection, and utilizes this breakpoint to generate localized recovery continuations for targeted fine-tuning.

As shown in the figure below:

To build the foundational graph structure, the framework introduces an Information Atom Abstraction mechanism. For a given task, useful facts extracted from tool responses are normalized into a task-specific set of information atoms $\kappa_x$κx​. This process operates in two stages. A deterministic stage canonicalizes each tool response to determine if it carries usable data, collapsing non-informative responses into a single class. A semantic stage then assigns atoms to the surviving candidates, ensuring that semantically equivalent responses share the same atom. The acquisition increment at each step is calculated based on newly acquired atoms, updating the information set.

The interaction state at any step is represented as a tuple $(I_t, U_t)$(It​,Ut​) comprising the set of acquired information atoms and the multiset of useless operations performed since the last main node. The root state is defined as a main node. Subsequent states are classified as main nodes if they acquire new information, or auxiliary nodes if they only record useless operations without changing the information set. The ISTG is constructed as a directed multigraph $G_x = (V_x, E_x)$Gx​=(Vx​,Ex​) from all successful and failed teacher rollouts, where main nodes form the information-acquisition backbone and auxiliary nodes represent attached useless exploration. This set-valued information component allows order-independent acquisition paths to reconverge at the same main information state.

Once the ISTG is established, the framework identifies the CTB for failed student rollouts. The authors define an empirical success-reachable region $\mathcal{R}_x^+$Rx+​ based on a task-specific reachability budget, which filters nodes on successful teacher rollouts whose remaining distance to success is within the budget limit. For each student state, a type-specific state projection is applied. For student main nodes, acceptable teacher anchors are reachable teacher main nodes whose information sets are contained within the student's acquired information. For student auxiliary nodes, the projection compares the number of useless operations performed after the corresponding reachable teacher main node. The CTB is formally defined as the first transition from a projectable to a non-projectable student state, marking the earliest point where the student departs from the teacher-supported region.

Following the identification of the CTB, the framework applies CTB-Guided Localized Supervision. The authors retrieve privileged references from successful and failed teacher traces. Instead of directly concatenating these traces, an augmented generator is conditioned on the task, the retained student prefix before the CTB, and the privileged references to generate a recovery continuation. The training trajectory is formed by appending this continuation to the retained prefix. The model is then trained using a masked causal language modeling objective $\mathcal{L}_{\text{DART}}$LDART​. Crucially, the loss mask is defined over response steps such that only tokens belonging to assistant response steps generated after the CTB and before the final-answer step receive non-zero weight. This ensures that supervision is strictly restricted to post-CTB recovery behaviors while preserving pre-CTB context.

Finally, the framework employs a Progressive Self-Distillation Paradigm. The task-specific ISTG is maintained throughout the iterative self-distillation process. At each iteration, the current student produces new rollouts that are mapped into the shared interaction-state space to detect updated CTBs. As the student policy improves, its states remain projectable for longer portions of the trajectory. Consequently, the detected CTBs track the evolving boundary where the current policy first departs from the teacher-supported region, inducing a self-paced curriculum that progressively shifts supervision toward recovery behaviors beyond the current tool-use capability.

Experiment

The evaluation setup trains DART-SD on the FTRL dataset, which contains over 2,000 tool-use environments with four task structures, and tests it against distillation-based and reinforcement learning baselines on both in-domain and four out-of-domain benchmarks under a no-thinking configuration. The results show that DART-SD consistently outperforms all compared methods and maintains strong performance across different model sizes, confirming that its topology-aware localized tuning learns generalizable tool-use capabilities rather than overfitting to the training distribution. Qualitative analysis reveals that the student can surpass its larger teacher model by discovering optimized shortcuts, producing successful trajectories that are shorter than the golden references, while tracking the capability boundary position shows that the first departure from recoverable behavior is progressively delayed across iterations. Ablation studies further validate that each component, including self-distillation, CTB-guided localized supervision, progressive iterative adaptation, and the structured ISTG topology, contributes to the overall effectiveness.

DART-SD consistently outperforms distillation-based and reinforcement learning methods on in-domain and four out-of-domain tool-use benchmarks for both Qwen3-4B and Qwen3-8B backbones, achieving the highest average performance. It surpasses the teacher model on several tasks while simultaneously learning more efficient strategies, reducing average tool calls over training iterations, and preserving general reasoning and instruction-following capabilities. DART-SD achieves the strongest overall performance across all five benchmarks, with the highest average scores for both Qwen3-4B and Qwen3-8B backbones. Training with CTB-guided localized supervision progressively shortens successful tool-use trajectories while improving task success, indicating more efficient reasoning rather than relying on longer chains. Beyond tool-use, DART-SD preserves and improves general capabilities on IFEval, AIME24, AIME25, and MMLU, outperforming both the base model and standard SFT.

On the FTRL test set, DART-SD's successful trajectories become shorter across progressive iterations while Solve-F1 improves steadily. The overall average tool-call length ends below the golden reference trajectory length, indicating more efficient tool-use behavior. Gains are most pronounced for parallel and multi-step tasks, while single-step trajectories approach golden efficiency. Overall average tool-call length decreases across iterations and falls below the golden reference by Iter5. Solve-F1 improves consistently over the same iterations, showing that shorter traces coincide with better task success. Parallel multi-step tasks show the largest efficiency gains and end below their golden reference length, while single-step and parallel single-step tasks remain close to golden efficiency.

Across progressive SFT iterations, the average CTB position in failed trajectories moves later, meaning models execute longer valid prefixes before needing localized recovery. The shift is strongest for multi-step and parallel multi-step tasks, while single-step and parallel single-step tasks remain relatively low or inconsistent. This supports the idea that CTB-guided localized supervision progressively extends the capability boundary for more complex trajectories. The overall CTB position improves substantially from the first to the second iteration, then continues to increase more gradually in later iterations. Multi-step and parallel multi-step tasks drive the overall trend, ending with the highest CTB positions among all task types. Single-step tasks show limited or non-monotonic boundary movement, with final values close to or below earlier peaks.

Under the thinking setting, DART-SD achieves the highest performance across all three reported benchmarks, outperforming the Qwen3-8B base model and all compared training baselines. It shows clear improvement over the leading baseline on FTRL and BFCL while maintaining a competitive ToolHop score, indicating that CTB-guided localized supervision remains effective when explicit thinking is enabled. DART-SD ranks first among compared methods on FTRL Solve-F1, BFCL Multi-Turn, and ToolHop AC. Compared with MatchTIR, DART-SD delivers greater gains on FTRL and BFCL and performs similarly on ToolHop. Trainable baselines and the inference-time thinking base model consistently trail DART-SD across all reported thinking-setting metrics.

DART-SD outperforms both the pretrained Qwen3-8B baseline and standard SFT across all reported capability benchmarks, including instruction following, mathematical reasoning, and general knowledge. The average score improves from 43.92 for the base model to 49.89 for DART-SD, while SFT reaches only 44.18. The gains are broad, with a particularly large relative improvement on instruction following. DART-SD achieves the best result on every individual benchmark and the highest overall average. Standard SFT provides only a slight average gain over the base model and underperforms DART-SD across all metrics. The largest relative gain for DART-SD appears on instruction following, where it improves substantially over both baselines.

The experiments assess DART-SD across tool-use benchmarks, trajectory efficiency, failure boundary location, thinking-mode performance, and general capability benchmarks. DART-SD outperforms distillation-based and reinforcement learning baselines on in-domain and four out-of-domain tool-use tasks for both Qwen3-4B and Qwen3-8B, surpasses the teacher model on several tasks, and learns shorter successful trajectories while improving task success. Its CTB-guided localized supervision also shifts failure boundaries later in complex multi-step tasks, remains effective under explicit thinking, and preserves or improves instruction following, mathematical reasoning, and general knowledge compared with the base model and standard SFT.