Command Palette
Search for a command to run...
Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training
Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training
Tingyun Li Wenfeng Feng Weiqing Li Abudukelimu Wuerkaixi Guohua Liu Yuewei Zhang
Abstract
Large language models ofer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Autonomous systems automate parts of this process by proposing updates, training candidates, and using evaluation feedback to select subsequent proposals. As evidence accumulates, a central problem emerges: which past update evidence remains actionable after subsequent training has changed the parent model? An update’s efect depends on its parent, data, and training stage. Treating past success as context-free permission can waste compute. If the resulting child is promoted, it can also degrade the subsequent training trajectory. We formulate this problem as conditional experience transfer and introduce Boundary-Calibrated Intervention Transfer (BCIT), a method that authorizes experience reuse before weight-changing training. BCIT binds an observed efect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and obtains current-state evidence through a bounded training trial when needed. Fully trained candidates still face a shared adoption rule, and only observed events extend memory. On one 4B model adapted across finance reasoning, text-to-SQL, and function calling, candidate updates exhibit heterogeneous target and retention efects across the evaluated contexts. Under matched candidates, evidence, and compute, BCIT authorizes fewer harmful updates and attains higher equal-budget finalmodel quality than the evaluated alternatives. These results support treating experience authorization as a distinct problem in autonomous post-training.
One-sentence Summary
Alibaba Cloud Computing proposes Boundary-Calibrated Intervention Transfer (BCIT), a conditional experience transfer method for autonomous LLM post-training that verifies reuse conditions, vetoes hard conflicts, and obtains current-state evidence through bounded trials, thereby reducing harmful updates and improving final model quality on a 4B model across finance reasoning, text-to-SQL, and function calling tasks.
Key Contributions
- The paper formulates conditional experience transfer as a distinct problem in autonomous LLM post-training, where past update successes are not universally actionable after the parent model has changed.
- It introduces Boundary-Calibrated Intervention Transfer (BCIT), a method that binds observed effects to source contexts, checks applicability conditions, vetoes hard conflicts, and uses bounded training trials to gather current-state evidence before authorizing full training.
- Experiments on a 4B model adapted across finance reasoning, text-to-SQL, and function calling show that BCIT authorizes fewer harmful updates and achieves higher equal-budget final-model quality than evaluated alternatives, supporting experience-conditioned compute allocation.
Introduction
In many continual learning and multi-task settings, models are updated sequentially with post-training compute, but improvements from one context can inadvertently harm performance on others. Prior approaches often allocate compute unconditionally, assuming past success justifies further modification, which risks introducing conflicts and degrading overall capability. The authors introduce BCIT, a method that conditions post-training compute allocation on evidence from the current context. It rejects hard conflicts, validates candidate updates against the parent model, and applies a frozen rule to grant full training budgets, then uses a policy-shared adoption rule to promote or roll back each child model. This evidence-conditioned strategy authorizes fewer harmful updates while preserving beneficial ones, yielding higher cross-task performance than shared-model alternatives.
Experiment
The BCIT policy achieves the strongest trade-off, authorizing far fewer harmful outcomes than any other policy while still preserving most beneficial outcomes. Removing candidate-specific applicability from BCIT causes its harmful authorization rate to triple, reverting to the level of the source-evidence-only baseline. Policies like additive+veto also reduce harmful authorizations but sacrifice more beneficial outcomes than BCIT. BCIT authorizes only a quarter of harmful outcomes while retaining 90% of beneficial ones, yielding the lowest harmful share. Without candidate-specific applicability, BCIT's harmful authorization rate jumps to 75%, matching the source-evidence-only policy. Source-evidence-only keeps 90% of beneficial outcomes but still authorizes three-quarters of harmful outcomes. Flat-additive and applicability-only policies both keep 80% of beneficial outcomes and authorize nearly two-thirds of harmful outcomes.
Under a fixed 36 GPU-hour budget, the Add. + veto method achieves the highest cross-task average and the largest minimum improvement over the base model, while also delivering the best instruction-following scores among shared models. Single-task specialists surpass shared models on their respective tasks but consume the full budget per task, making them computationally expensive. The multi-task approaches consistently improve over the base model across all tasks without exceeding the budget. Add. + veto yields the highest cross-task mean (46.1) and the largest minimum gain over the base model (+2.7 points) among shared-model methods. Single-task specialists reach 39.6 on TAT-QA, 49.3 on BIRD, and 65.1 on BFCL, outperforming the best shared model but requiring 36 GPU-hours each. Instruction-following (IFEval) scores are highest for Add. + veto (76.0 prompt-level, 82.1 instruction-level) compared to other shared models.
In safety evaluations, the BCIT policy achieves the best trade-off between harmful and beneficial authorizations, drastically reducing harmful outcomes while preserving most beneficial ones; this advantage depends on candidate-specific applicability, without which harmful authorizations rise sharply. In multi-task training under a fixed compute budget, the Add. + veto method yields the highest cross-task average and instruction-following scores among shared models, while single-task specialists achieve higher per-task performance but at much greater computational cost.