One-sentence Summary

Amazon researchers introduce Rufus-Air, an open and reproducible post-training recipe for GLM-4.5-Air-Base that applies a serial pipeline of SFT, reasoning RL, coding RL, instruction-following RL, general agent, coding agent, search agent, and RLHF stages using public data without new annotations, and it improves over the official GLM-4.5-Air release while remaining competitive with similarly sized open models.

Key Contributions

• Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-Base (106B-A12B) organized as an eight-stage serial pipeline: SFT, reasoning RL, coding RL, instruction-following RL, general agent, coding agent, search agent, and RLHF. It builds on open-source components and public data, much of it used as released, with no new human annotation or in-house distillation teacher.
• The recipe is organized around four principles: diverse, high-quality SFT establishes a strong capability floor; difficulty filtering keeps RL prompts within a productive learning range; and stage ordering follows reward reliability from hard verifiable rewards to softer judge-based signals. Infrastructure choices such as token-in/token-out rollouts, consistent chat templates, reliable sandboxing, and large batches with Rollout Routing Replay are treated as part of the recipe rather than implementation details.
• Rufus-Air leads the official GLM-4.5-Air release on every reported benchmark except Arena-Hard v2 Creative Writing and is competitive with similarly sized open models, including INTELLECT-3 and Nemotron-3-Super.

Introduction

Open-weight base models have made post-training research more accessible, but the actual training recipes are usually reported thinly, often reading more like system cards than reproducible procedures. The missing details around data filtering, stage ordering, and infrastructure make it hard for other teams to replicate or build on published results. The authors address this gap by presenting Rufus-Air, a fully documented post-training recipe built on GLM-4.5-Air-Base using open-source components and a compute footprint feasible outside frontier labs. Their main contribution is the reusable recipe itself, organized around key decisions such as treating supervised fine-tuning as a capability-building stage, filtering prompts by difficulty, ordering RL stages by reward reliability, and making infrastructure details part of the method.

Dataset

Dataset Description

Supervised Fine-Tuning

• Sources and composition: The SFT mix contains 9.01M samples and 44.5B raw tokens, with 66.7M conversational turns and 23.5M supervised turns. After masking system messages, user turns, and tool observations, 27.0B tokens, or 60.8%, contribute to training loss. The data is grouped into six capability categories: General Agent, General Chat, STEM, Math, Code, and Coding Agent. Sample count and training-token share differ substantially because reasoning traces and multi-turn coding trajectories are much longer. All prompts and responses come from 17 public datasets. The authors do not perform a new regeneration pass or commission new human annotations.
• Preprocessing: Every source is normalized into a role-aware multi-turn schema with per-message loss masks. Invalid records and records without a supervised assistant response are dropped. Agent trajectories are converted into interleaved thought, tool-call, and observation chains. Assistant turns are supervised only when they form part of a complete response after the last real user request; earlier assistant rationales are stripped and masked. System, user, and tool-observation messages receive zero loss.
• Validation and length filtering: Rule-based validators check balanced reasoning tags, valid role transitions, and masking around tool interactions. Sequence length is measured after applying the production chat template, including role markers, reasoning wrappers, tool schemas, calls, and observations, and is capped at 120K tokens.
• Decontamination: The SFT mix is screened against reported benchmarks using word-level 8-gram overlap, with additional exhaustive n-gram and dense retrieval screens for math and knowledge benchmarks. The chained benchmark passes remove 3,529 samples, while the additional detectors and manual review bring total removals to 4,187 unique instances.
• Usage: The model is trained from GLM-4.5-Air-Base for three epochs over the full 9.01M-example mixture, with packing to 128K token contexts. Only assistant tokens contribute to the token-level loss. Checkpoint 3799, selected from the first training plateau, initializes the subsequent RL stages.

Reasoning RL

• Sources and composition: The reasoning prompt set combines public reasoning problems with reference solutions, synthesized and augmented data generated through Enigmata and ReasoningGym, and hard third-party math sets. Problems are auto-tagged into fine-grained domains and deduplicated. The prompt set spans three verifiable single-turn task families. Math dominates prompt counts, while logic puzzles dominate prompt tokens, averaging about 688 tokens per prompt across 134 task generators in 15 reasoning categories.
• Processing: The same contamination screening used for SFT is applied. A correctness filter removes prompts for which the teacher model, GPT-OSS-120B, cannot obtain positive reward. A learnability filter then keeps prompts with a current-policy solve rate above 0 and at most 0.8, removing prompts that are too easy or currently unlearnable. Online dynamic sampling further keeps only groups with average reward in the productive band.
• Usage: The filtered prompts are used as verifiable single-turn RL training prompts. Most prompts, 83.5%, request a boxed answer that is matched against a short gold label. Rollouts sample 256 prompts with 16 responses per prompt under a 30,000-token response budget.

Coding RL

• Sources and composition: The coding problem set aggregates four sources: EvolveCoder, Nemotron, Dolci, and ADR. It mixes functional problems, which carry assert-based unit tests, with stdin/stdout problems, which carry input/output string pairs. Codeforces is the largest single platform, representing about 8,700 problems, or roughly 30% of the set.
• Processing: Records without a non-empty test payload are dropped. Deduplication uses the SHA-256 hash of the whitespace-normalized, lowercased problem statement. Leakage is checked against LiveCodeBench v6 using exact prefix hashes and character-level 60-gram overlap; no training problem matches an evaluation problem. A teacher-based solvability screen retains only problems where at least one sampled GPT-OSS-120B completion passes the verifier. A difficulty filter removes problems solved by all four base-policy warm-up samples, effectively dropping easy prompts.
• Usage: The filtered problems are used with binary execution-based rewards. Per-problem tests are deterministically subsampled to at most 50. Each rollout starts with 128 prompts and 64 samples per prompt, later adjusted to 64 prompts after extending the response budget to 128K tokens.

Instruction-Following RL

• Multi-constraint IF data: The authors synthesize 14K single-turn prompts whose constraints are all checkable by Python. A teacher LLM, Qwen3-235B-A22B, expands a manually written pool of single-constraint instructions into prompts with 2 to 6 atomic constraints each and generates a Python verification function per constraint. Difficulty filtering removes prompts the policy already solves at a rate above 80%, and quality filtering removes prompts the teacher itself cannot pass in four attempts.
• Multi-turn conversational IF: The authors adversarially synthesize multi-turn conversations. A teacher LLM plays the user and tries to make assistant models fail by layering instructions, revising earlier requirements, and introducing distractors. Different open-weight models take the assistant role across examples. A portion of the data includes a system prompt for instruction-hierarchy training. The final dataset contains 13K conversations, averaging 6.1 turns and 2.7 rubrics each.
• Processing: Three filters are applied to the multi-turn data: sanity filtering removes overlong conversations and over-refusal cases, quality filtering removes examples the teacher cannot pass in four attempts, and difficulty filtering removes prompts the policy already solves too easily.
• Usage: The two datasets are mixed and trained jointly in one stage. Rewards are rubric-based and binary: all rubrics must be satisfied for positive reward. Rubrics are either code-verifiable Python checks or LLM-judge checks. The authors deliberately exclude optional rubrics to avoid rewarding length bias.

Coding Agent

• Sources and composition: Training tasks come from Endless-Terminal, SETA-Env, and Scale-SWE. Endless-Terminal provides procedurally generated containerized terminal tasks with completion tests. SETA-Env provides more than 4,500 verified Harbor-format terminal environments synthesized from web sources. Scale-SWE supplies executable software-engineering tasks built from public GitHub pull requests, including Docker images and fail-to-pass unit tests.
• Processing: Structural filters remove tasks unsuitable for scaled execution, such as those with missing files or multi-container compositions. A learnability filter removes tasks the current policy solves reliably or never solves, leaving about 4K training tasks. A further roughly 10K Scale-SWE tasks remain available but are not used.
• Usage: Tasks are converted to a unified Harbor format containing an instruction file, an environment Dockerfile, and a test verifier. The agent uses a single shell-command execution tool. Reward is the task verifier reduced to a binary outcome at termination.

Search Agent

• Sources and composition: Verifiable question-answer pairs are extracted from MiroVerse. Questions requiring vision are dropped, and source subsets that a proxy policy almost never solves are excluded. This yields 36,614 candidates, with 4,069 additional items held out for validation.
• Processing: The candidate set is narrowed in two passes. The first pass removes questions answered correctly in any of 8 tool-free rollouts, since those are solvable from parametric knowledge alone. The second pass uses 8 full-tool rollouts per question with a judge-graded reward and keeps questions with a raw correct count between 0 and 4. This leaves a final RL training set of about 2.2K questions.
• Usage: The filtered questions train a research-style web agent. The agent may use web search, web scraping, and a sandboxed Python interpreter, with up to 100 tool calls during training. Final answers must be boxed. The reward uses an LLM judge with partial credit and requires at least two successful tool calls for positive reward.

RLHF

• Sources and composition: The authors consider three public preference datasets as prompt sources: Arena Human Preference, HelpSteer3, and HH-RLHF. They filter out examples with missing, empty, or otherwise degenerate responses and score the remaining responses with Skywork-Reward-V2-Qwen3-8B. HH-RLHF has much shorter responses than Arena Human Preference, while HelpSteer3 has shorter responses but the largest average score separation between chosen and rejected responses.
• Processing and usage: The final RLHF stage uses 75,815 HH-RLHF prompts remaining after validity filtering. RLHF training uses only the prompts. The chosen and rejected responses and preference labels are used only for offline dataset analysis. No new human labels are introduced. The open reward model scores the policy’s own on-policy rollouts during training.

Method

The authors design a serial post-training pipeline that transforms the GLM-4.5-Air-Base Mixture-of-Experts model into the final Rufus-Air checkpoint. The pipeline consists of eight sequential stages, each building upon the checkpoint produced by the previous one. The stages progress from basic capability building and hard verifiable rewards to advanced tool use and softer preference alignment.

The first stage is Supervised Fine-Tuning (SFT), which establishes broad coverage across chat, mathematics, STEM, coding, and tool use while normalizing the output format. The authors construct a diverse SFT mixture containing 9.01 million samples and 44.5 billion raw tokens. The data is grouped into six capability categories, and the distribution of samples versus loss-contributing tokens differs substantially across these categories.

During SFT training, the model is optimized for three epochs using AdamW with a cosine learning rate schedule. The training loss decreases steadily across epochs, while held-out evaluation metrics plateau early, indicating that the model acquires the necessary foundational capabilities quickly. The authors select an intermediate checkpoint from this plateau to initialize the subsequent reinforcement learning stages.

Following SFT, the pipeline applies Reinforcement Learning with Verifiable Rewards to enhance specific capabilities. Reasoning RL targets math and scientific reasoning using deterministic verifiers and Group Sequence Policy Optimization. The authors employ strict data filtering to ensure prompts are both solvable and within the productive learning band of the policy. Coding RL then improves competitive programming performance using execution-based rewards. The training dynamics show a steady increase in both the training reward and the pass rate on coding benchmarks, with a mid-run extension of the response budget from 64K to 128K tokens to eliminate truncation masking and further boost performance.

Instruction-Following RL is the next stage, designed to improve the model's ability to adhere to complex, multi-constraint instructions and retain layered instructions across multi-turn conversations. The authors synthesize datasets for both single-turn and multi-turn instruction following, utilizing a rubric-based binary reward system where rewards are granted only when all necessary conditions are satisfied. Training with Group Relative Policy Optimization leads to a consistent rise in the training reward and evaluation scores, while the response length decreases as the model learns to follow instructions precisely without unnecessary padding.

The subsequent three stages introduce tool use in specialized environments. General Agent training establishes a broad habit of tool orchestration using synthetic Model Context Protocol tasks. Coding Agent training focuses on terminal and software engineering tasks within sandboxed environments, utilizing execution-based verifiers. Search Agent training targets multi-hop factual questions over the open web, relying on an LLM judge for outcome-based rewards and employing mean-only group advantages to handle the noisy reward signal. Finally, RLHF shapes open-ended quality using a learned reward model and on-policy RL, optimizing for concise, helpful, and harmless responses to mitigate reward hacking risks.

Experiment

The evaluation compares Rufus-Air with open-weight models of similar active-parameter scale, including two post-trainings of the same base checkpoint and one model from a different base, across instruction following, reasoning and knowledge, and general, search, and coding agent benchmarks, while also tracing how capabilities accumulate across the eight-stage training pipeline. The main results show that Rufus-Air leads the same-base baselines on nearly all rows and leads the stronger different-base baseline on instruction following and most agentic tasks, with gains concentrated where the recipe spends its training signal, while mathematics and science remain close and creative writing is a notable exception. Stage-by-stage results indicate that each stage primarily moves its own target, with SFT already providing a strong pre-RL floor, and the discussion highlights learnability filtering, stage ordering by reward vulnerability, infrastructure choices, and the extra cost of agentic RL as key cross-stage findings. The authors also bound the conclusions by noting that the sequential stage order is not proven optimal and that Coding Agent was trained only to the available compute budget.

Rufus-Air leads on instruction-following and most alignment and agentic benchmarks among models evaluated under the same harness, while staying close on competition mathematics and knowledge. The main exceptions are Tau2-Airline, Tau2-Retail, and Arena-Hard v2 Creative Writing, where comparison models remain ahead. Gains are largest in areas that receive the most training signal. Rufus-Air leads on instruction-following and most alignment and agentic benchmarks among the four models evaluated under a common harness. Nemotron-3-Super remains ahead on Tau2-Airline and Tau2-Retail, and both Nemotron-3-Super and GLM-4.5-Air lead on Arena-Hard v2 Creative Writing. On competition mathematics and knowledge benchmarks, Rufus-Air performs close to the other models, and it ties Nemotron-3-Super on Terminal-Bench 2.1. The strongest gains appear in areas where the recipe spends the most training signal.

The pipeline shows stagewise capability accumulation: supervised fine-tuning improves math benchmarks but lowers GPQA, then reasoning RL recovers GPQA while slightly reducing math scores. Coding RL then lifts LiveCodeBench, and instruction-following RL produces the largest targeted gains on IFBench and Multi-challenge. Agent stages later improve tool-use and software-engineering benchmarks, and RLHF improves both Arena-Hard splits, especially creative writing. Supervised fine-tuning raises AIME scores above the base model but leaves GPQA below it; reasoning RL restores GPQA close to the base level while modestly lowering AIME. Instruction-following RL builds on the coding checkpoint and delivers the largest relative gains on IFBench and Multi-challenge, with a smaller further gain on IFEval. After reasoning RL, the next two stages keep GPQA and AIME 26 nearly unchanged while adding coding and instruction-following skills. General agent and coding agent stages improve their target benchmarks, including MCP-Atlas, Tau2-Retail, Terminal-Bench, and SWE-bench Verified.

The SFT mixture spans six capability categories and shows a large mismatch between sample share and loss-contributing token share. General Agent is the largest category by sample count, while Math and Coding Agent dominate token share because their reasoning traces and multi-turn trajectories are much longer. Sample share and training-token share diverge sharply across categories, so sample counts capture source coverage while token shares reflect where supervision effort is concentrated. Math and Coding Agent together account for a minority of samples but nearly half of assistant training tokens.

Before any RL, the SFT-only checkpoint already outperforms the public GLM-4.5-Air release on instruction following and both AIME benchmarks, with a notably large lead on IFBench. It trails only on GPQA, which is the gap the later RL stages aim to close. This makes the SFT checkpoint a competitive starting floor for RL rather than only a warm-up. The SFT-only checkpoint surpasses the RL-included public release on IFEval, IFBench, AIME 25, and AIME 26. GPQA is the only reported metric where the SFT checkpoint falls behind, and RL stages later target that gap.

The RLVR prompt set combines math, science, and puzzle tasks, each with a different verifier. Math provides the largest share of prompts, while puzzles are the smallest family by prompt count but account for the majority of prompt tokens because their statements are much longer. Science sits between the other two in prompt count and has the lowest token share. Math contributes just under half of prompts, science about a third, and puzzles roughly one in six prompts. Puzzles account for more than half of all prompt tokens despite their smaller prompt share, with puzzle statements roughly six times longer than the others. Each family uses a distinct verifier: canonical answer matching for math, fuzzy string matching for science, and generated Python checkers for puzzles.

The experiments evaluate Rufus-Air against peer models under a common harness and trace capability changes across training stages. Rufus-Air leads on instruction following and most alignment and agentic benchmarks while remaining close on math and knowledge, with Tau2-Airline, Tau2-Retail, and creative writing as main exceptions. Stagewise results show supervised fine-tuning improves math but lowers GPQA, reasoning RL recovers GPQA, coding and instruction-following RL add targeted skills, and later agent stages improve tool use and software engineering. The SFT-only checkpoint already surpasses the public GLM-4.5-Air release on instruction following and AIME benchmarks, and data analyses reveal large mismatches between sample share and token share in both SFT and RLVR prompt mixtures.