ThinkingBox Converts Database Grading into RL Rewards for Post-Training Agents

Microsoft and Hugging Face have released ThinkingBox-Training on GitHub, presenting a follow-up to their benchmark showing how executable database-state checks can train AI agents that complete enterprise workflows reliably. The work builds on ThinkingBox-Bench, a 507-task benchmark that evaluates agent success by inspecting the final database state rather than relying on tool-call traces or self-reported completion. The key methodological shift is using those terminal-state checks as reinforcement-learning rewards. For each task, an agent works against an isolated backend with tools and a simulated user; when the episode ends, deterministic checks compare the resulting records with expected outcomes. A correct state earns a reward of 1, while incorrect, missing, or extra side effects earn 0. The authors then selected training tasks by estimating each task’s success probability under the base model, keeping only tasks with enough uncertainty to produce informative contrast during training. This offline policy-relative difficulty approach produced a 187-task pool for fine-tuning. Fine-tuning Qwen3.8-27B with 50 steps of Group Relative Policy Optimization improved average single-attempt success on ThinkingBox-Bench from 51.70% to 60.78%. More strikingly, the number of tasks passed in all 20 evaluation runs rose from 38 to 107, increasing fully reliable task coverage from 7.50% to 21.10%. Tasks solved at least once slipped slightly, from 453 to 446, suggesting training sharpened repeatable behavior rather than broadly expanding solution coverage. Domain results showed gains in retail, auto insurance, travel, neobank, and consulting, with the largest consistency improvement in retail. The post also frames reliability in economic terms. Estimated evaluation cost per successful attempt fell from $0.177 to $0.139, and cost per task passed every time dropped from $24.36 to $8.00, bringing the 27B fine-tuned model closer to several larger commercial models on dependability pricing. The authors caution that pass@1 alone can obscure flaky behavior, recommending reports of both at-least-once coverage and all-attempts consistency. They also stress validating that training and evaluation systems interpret model outputs identically, especially tool names and arguments. The release provides code for data selection, rollout, reward computation, and Verl integration, while noting ThinkingBox tasks are synthetic and benchmark data remains separate from training.