One-sentence Summary

In their introduction of AgentInstruct, an extensible agentic framework that automatically generates diverse, high-quality synthetic post-training data from raw seeds to produce 25M prompt-response pairs, researchers at Microsoft Research demonstrate that post-training Mistral-7b with this data yields Orca-3, which achieves significant improvements over Mistral-7b-Instruct across benchmarks—$40\%$40% improvement on AGIEval, $19\%$19% improvement on MMLU, $54\%$54% improvement on GSM8K, $38\%$38% improvement on BBH, and $45\%$45% improvement on AlpacaEval—while consistently outperforming LLAMA-8B-instruct and GPT-3.5-turbo.

Key Contributions

• The paper introduces AgentInstruct, an extensible agentic framework that automatically generates large-scale synthetic post-training data by producing both prompts and responses from raw seeds such as unstructured text documents and source code, using a taxonomy of over 100 subcategories and agents equipped with powerful LLMs, tools, and reflection flows.

• This work demonstrates the framework's scale by creating a 25 million pair post-training dataset covering skills such as text editing, creative writing, tool usage, coding, reading comprehension, reasoning, math, and RAG, with only publicly available raw materials as seeds and no task-specific or benchmark data.

• Results show that post-training Mistral-7B with this dataset yields the Orca-3 model, which improves over Mistral-7B-Instruct by 40% on AGIEval, 19% on MMLU, 54% on GSM8K, 38% on BBH, and 45% on AlpacaEval, reduces hallucination by 31.34% across summarization benchmarks, and consistently outperforms LLAMA-8B-instruct and GPT-3.5-turbo.

Introduction

Synthetic data has become a key accelerator in LLM training across stages like pre-training, instruction-tuning, and RLHF, yet generating data that is both high quality and diverse remains difficult. Prior work often relies on powerful teacher models to produce responses to existing prompt sets, but this can lead to model collapse, stylistic imitation rather than capability gain, and heavy dependence on human curation and filtering. The authors address these limitations with AgentInstruct, an agentic solution for what they call Generative Teaching, which uses multi-agent workflows with reflection, tools, and a taxonomy of over 100 subcategories to generate both prompts and responses autonomously. Instead of requiring seed prompts, AgentInstruct starts from raw unstructured documents, which are abundant and promote learning general capabilities rather than benchmark-specific behaviors. The authors demonstrate its utility by generating a 25 million pair post-training dataset that, when used to fine-tune Mistral-7B, produces the Orca-3 model with substantial gains across benchmarks such as AGIEval, MMLU, GSM8K, and BBH, plus reduced hallucination, all without using any task-specific or benchmark seed data.

Dataset

The authors build the Orca-3 training corpus from two main components.

Main corpus composition

• Approximately 22 million instructions generated via the AgentInstruct Flow, which covers 17 distinct capabilities (detailed in their Table 1).
• Seed content for this generation comes from unstructured text and code files sampled from KnowledgePile, AutoMathText, a subset of OpenStax, and a subset of Apache-2.0 licensed source code files.
• Using unstructured content as seeds avoids any benchmark-specific data, letting the team optimize for general capability rather than a particular evaluation set.
• An additional 3.8 million paired instructions are sourced from Orca-1, Orca-2, Orca-Math, and samples from other public sources. This portion is referred to as the Orca-2.5-dataset.
• The combined corpus totals approximately 25.8 million paired instructions, all used to train Orca-3. A separate model, Orca-2.5, is trained only on the 3.8 million instruction subset to isolate the impact of the AgentInstruct-generated data.

Training usage and processing details

• The 25.8 million paired instructions finetune Mistral-7b-v0.1, chosen for its public base weights and permissive license.
• Each pair is tokenized with the Mistral tokenizer, capped at a maximum sequence length of 8192 with packing.
• Label masking ensures the training loss is computed only on the response conditioned on the prompt.

Evaluation datasets

• Orca-Bench serves as a held-out test set, with 100 samples per skill for all 17 curated capabilities.
• The Open Domain Question Answering (ODQA) category has two test subsets: ODQA, with 100 questions from the initial seed instruction phase, and Complex ODQA, with more intricate questions from the refinement phase.
• Entries in Orca-Bench may involve multiple exchanges. Each student response is evaluated against the teacher (GPT-4) response on a 0 to 10 scale, with the final score normalized by the teacher's total.
• Orca-Sum is a new benchmark for summarization and grounded data transformation. It samples data from 45 Hugging Face summarization datasets across domains such as news, conversations, science, health, social, e-mails, and code, yielding 458 datapoints. Up to 1,000 candidate points are collected and carefully deduplicated to avoid overlap with training data. GPT-4 generates 40 dataset-specific prompts per dataset, from which one is randomly selected per datapoint, forcing the model to move between formats and domains.

Method

The authors propose AgentInstruct, a systematic pipeline for generating high-quality synthetic instruction-tuning data at scale. As shown in the figure below:

The core architecture relies on agentic flows that automate data synthesis while ensuring diversity and complexity across various reasoning and tool-use capabilities. The pipeline operates through three distinct stages: Content Transformation, Seed Instruction Generation, and Instruction Refinement.

In the initial Content Transformation Flow, raw seeds such as web articles, textbook chapters, or source code snippets are processed to create intermediate representations optimized for downstream task generation. A collection of specialized agents evaluates each seed and applies stylistic or structural modifications, such as converting factual text into argumentative passages or debates. This step not only standardizes the input format but also introduces substantial variation in the resulting data distribution.

Following transformation, the Seed Instruction Generation Flow takes the modified seeds and produces a diverse set of initial instructions. The system employs a predefined taxonomy to guide agent selection, ensuring coverage across multiple cognitive and functional categories. For reading comprehension, agents generate questions targeting literal understanding, critical analysis, or logical inference. For tool-use scenarios, the flow synthesizes API descriptions from code snippets and formulates tasks that require single or multi-step API calls, including scenarios with missing or superfluous parameters. Each agent is designed with a specific system prompt and optional tool-use capabilities to execute its designated role.

The Instruction Refinement Flow then iteratively enhances the quality and difficulty of the generated instructions. This stage utilizes a Suggester-Editor agent pair. The Suggester agent analyzes the current instruction and proposes modifications to increase complexity, such as introducing unanswerable conditions, altering ground-truth answers, or adding challenging distractors. The Editor agent subsequently implements these suggestions, producing a refined instruction-response pair. This iterative loop continues until the desired level of intricacy is achieved, effectively simulating a human-in-the-loop curation process without manual intervention.

To train the final model, the authors leverage a dataset comprising 25.8 million instruction-response pairs generated through this pipeline. The base architecture is Mistral-7b-v0.1, which undergoes supervised fine-tuning on the synthetic data. Each pair is tokenized with a maximum sequence length of 8192 and packed to optimize GPU memory utilization. Label masking is applied during the forward pass to ensure that the cross-entropy loss is computed exclusively over the model's generated responses rather than the prompts. The training configuration utilizes the AdamW optimizer with a weight decay of 0.1. The learning rate is initialized at $8 \times 10^{-6}$8×10−6 and follows a cosine decay schedule, preceded by a linear warmup over the first 500 steps. The optimization process runs for three epochs across a cluster of 152 NVIDIA A100 GPUs with a global batch size of 10, concluding after approximately 200 hours of computation.

Experiment

The evaluation setup positions Orca-3, a model post-trained with AgentInstruct synthesized data, against several baselines including Orca-2.5, Mistral-Instruct-7B, and GPT-4 across a wide range of benchmarks spanning reasoning, reading comprehension, math, format following, summarization, and retrieval-augmented generation. The results consistently show substantial gains from AgentInstruct training: notable improvements over baselines on the held-out Orca-Bench, significant boosts in reading comprehension that enable a 7B model to match GPT-4 on LSAT sections, large gains in math proficiency across difficulty levels, and improved format following that surpasses Gemini Pro. The approach also reduces summarization hallucinations by roughly a third while maintaining GPT-4-level quality, and markedly enhances RAG performance in the medical domain, demonstrating effective skill transfer to new domains.

The table lists 17 capabilities for which AgentInstruct Flows were implemented, covering skills such as reading comprehension, open domain QA, text modification, web agents, brain teasers, analytical reasoning, multiple choice questions, and data-to-text generation. These flows were used to create a large-scale synthetic dataset of about 22 million instructions from unstructured text and code sources, later combined with additional paired instructions to train Orca-3. The approach aims to enhance model capabilities across diverse tasks while avoiding benchmark-specific data. AgentInstruct Flows were built for 17 distinct capabilities, ranging from reading comprehension to data-to-text generation. The synthetic dataset was generated using unstructured text and code files as seeds, enabling large-scale and diverse instruction creation. The capability-focused approach avoids benchmark-specific data, optimizing for general skills rather than specific test performance.

On Orca-Bench, models trained with AgentInstruct data outperform existing baselines, with the final Orca-3 model achieving the highest average score. The gains are consistent across training checkpoints, showing progressive improvement with more training epochs. Orca-3 improves over the Orca-2.5 baseline by about 34% on average, and over Mistral-Instruct-7B by about 15%. The final Orca-3 model scores 9.55, notably above ChatGPT (8.13) and Mistral-Instruct-7B (8.31). Performance increases steadily from epoch 1 (9.35) to epoch 2 (9.49) to the final model (9.55), indicating continued gains from AgentInstruct data.

Orca-3 consistently outperforms its base model Mistral-7B-Instruct across most benchmarks, with the largest relative gains on GSM8K and AGIEval. It also surpasses larger or proprietary models like LLAMA3-8B and GPT-3.5-turbo on several tasks, while remaining competitive with GPT-4 on ARC and MMLU. The only notable regression is on GPQA, where Orca-3 slightly trails Mistral. Orca-3 shows the biggest relative improvement over Mistral on GSM8K, with a 54% gain, and also strong gains on AGIEval and BBH. Orca-3 exceeds GPT-3.5-turbo on MMLU, ARC, BBH, and DROP, while matching or approaching GPT-4 on ARC and MMLU. The only benchmark where Orca-3 falls below Mistral is GPQA, with a small relative decline of 4%.

Orca-3 consistently outperforms both Orca-2.5 and Mistral-7B-Instruct across reading comprehension benchmarks, with average improvements of 18% and 21% respectively. The largest gains are seen in AGIEval lsat-lr, where Orca-3 improves by 45% over Orca-2.5 and 36% over Mistral. Its average score also exceeds GPT-3.5-turbo, approaching GPT-4's level. Orca-3 achieves the highest average score among all compared models except GPT-4, surpassing GPT-3.5-turbo. The biggest relative gains are on AGIEval lsat-lr, with Orca-3 improving by 45% over Orca-2.5 and 36% over Mistral-7B-Instruct. On DROP, Orca-3 shows a 22% improvement over Mistral-7B-Instruct, the second largest gap among tasks. Orca-3's gains are consistent across all five sub-tasks, with improvements over both baselines on each metric.

AgentInstruct-trained Orca-3 models consistently outperform both Orca-2.5 and Mistral-7B-Instruct across all math benchmarks, with the largest gains on multistep arithmetic and abstract algebra. Improvements range from roughly 12% to over 1400% relative to baseline models, demonstrating effective skill transfer across elementary to college-level math. Orca-3 shows dramatic relative gains on multistep arithmetic, improving by over 1400% versus Orca-2.5 and over 880% versus Mistral-7B-Instruct. On abstract algebra, Orca-3 nearly doubles the score of Orca-2.5 and more than triples Mistral-7B-Instruct, while also surpassing GPT-3.5-turbo. Orca-3 achieves the largest absolute score on high-school mathematics among all non-GPT-4 models, and matches GPT-4 on that benchmark. Across all listed benchmarks, Orca-3 outperforms its immediate predecessor Orca-2.5 by margins ranging from 12% to 1418%, and exceeds Mistral-7B-Instruct by 44% to 882%.

The evaluation setup used AgentInstruct Flows to generate a large synthetic dataset across 17 capabilities, which was then used to train Orca-3. Across multiple benchmarks, including Orca-Bench, reading comprehension, and math, Orca-3 consistently outperformed its base model and prior versions, often exceeding larger or proprietary models like GPT-3.5-turbo and approaching GPT-4. The gains were broad and progressive with training, with the only notable regression appearing on GPQA. Overall, the capability-focused data synthesis proved effective for improving general skills without relying on benchmark-specific data.