One-sentence Summary

AllSpark Team presents Iris-mini and Iris-pro, $35\text{B-A}3\text{B}$35B-A3B and $397\text{B-A}17\text{B}$397B-A17B search agents trained via a reverse-constructed multi-hop question pipeline with trajectory- and judge-level filtering, followed by alternating SFT and RL against live search (SFT–RL climbing), achieving $82.2/84.8/86.9/52.3$82.2/84.8/86.9/52.3 and $88.6/85.1/92.9/56.4$88.6/85.1/92.9/56.4 on BrowseComp, BrowseComp-ZH, DeepSearchQA, and Humanity’s Last Exam with context management, the strongest open-source results in their parameter ranges.

Key Contributions

• Presents Iris-mini and Iris-pro, two search agents trained at the 35B-A3B and 397B-A17B scales, with a data pipeline that reverse-constructs multi-hop questions from web hyperlink structure, rewrites answer entities into descriptive references to block string matching, and admits only questions a reference model fails closed-book but solves with evidence.
• Combines trajectory-level and turn-level filtering before supervised fine-tuning, then applies reinforcement learning against live search with an in-cluster judge and observation summarizer, alternating SFT and RL in a procedure called SFT–RL climbing that feeds the hardest solved and most efficient rollouts back into the next supervised pass.
• With context management enabled, achieves 82.2/84.8/86.9/52.3 and 88.6/85.1/92.9/56.4 on BrowseComp, BrowseComp-ZH, DeepSearchQA, and Humanity’s Last Exam, the strongest overall results among open-source search agents in their parameter ranges, and reports positive transfer of search-trained models to general tool use and cowork benchmarks.

Introduction

Search agents extend language models beyond closed-form reasoning by enabling interaction with external tools and dynamic environments, requiring decisions about what to search, how to interpret evidence, and when to stop. This differs from conventional language-model evaluation, where tasks and computation budgets are fixed, and building reliable agents demands stronger models, better training data, and long-horizon interaction strategies. Prior work has progressed through tool use, reinforcement learning with retrieval, and synthetic task construction, but performance often varies more due to inference-time harness differences than the underlying policy, particularly in context management (CM). Long-horizon searches can exhaust context windows, shrinking the effective budget, and existing solutions like trajectory summarization or history removal obscure how much performance stems from the policy versus the harness, making CM a critical yet underreported factor.

The authors present an end-to-end recipe for data construction, training, and evaluation of search agents. They build multi-hop tasks from web-graph structure, remove easily searchable anchors, and retain only questions that are difficult and objectively verifiable. They collect trajectories from a strong teacher, apply trajectory-level and turn-level filtering, then optimize the policy with reinforcement learning against live search, using an in-cluster judge and observation summarizer, alternating SFT and RL iteratively to reinforce successful behaviors. Their main contribution is a complete pipeline that combines filtered SFT, RL with live web search, and iterative climbing, evaluated under two inference regimes with and without CM, achieving best overall performance among open-source agents on benchmarks like BrowseComp, BrowseComp-ZH, DeepSearchQA, and Humanity’s Last Exam. They also plan to release model weights and pipeline components to facilitate reproduction and further research.

Dataset

Dataset Composition and Sources

The authors build a training dataset for a search agent using a fully LLM-driven pipeline that reverse-constructs questions from the hyperlink structure of a web corpus. The goal is to generate questions that (i) cannot be answered from parametric memory alone and (ii) require composing evidence from multiple sources. The pipeline has three stages: web-graph construction, task synthesis, and dual-criteria verification. The final dataset also includes some in-house and open-source question sets (referenced as [6, 28, 2, 25]).

Web-Graph Construction

• The corpus is modeled as a directed graph where nodes are pages and edges are hyperlinks.
• Each synthesis instance starts with a seed page sampled using an answer-anchored mode: the authors fix a target answer entity and retrieve pages describing it.
• The seed is expanded along its out-links into a local subgraph, retaining full page text truncated to a fixed budget.
• To recover the true out-link set, the authors merge two sources: a structured semantic mirror of the corpus (RDF triples) and rendered page markup. This merge maximizes link recall because reader services strip inline anchors.

Task Synthesis

Given a subgraph, the pipeline produces a single obscured multi-hop question in three steps:

1. Entity-graph extraction: Raw pages are distilled into a compact, connected entity graph (with at most n nodes) containing salient entities and typed semantic relations. The extractor keeps only entities and relations that lie on a multi-hop path toward the seed theme, creating a dense relational skeleton for question authoring.
2. Multi-hop question generation: The seed theme (target answer) is used to generate an initial question with a reasoning path. The path must have at least N coupled relations, and the answer is not allowed to appear in the question text.
3. Anchor abstraction: Every non-answer entity is rewritten into a descriptive reference that uniquely identifies the entity but removes its name and aliases. This forces the final question to require disambiguation-by-reasoning rather than string matching.

Dual-Criteria Verification

• A pair (abstracted question, answer) is admitted only if it passes two criteria judged by a reference model:
  • Difficulty: The reference model, in closed-book mode with no tools, cannot answer the question correctly.
  • Solvability: The reference model, given the entity graph as context, can answer the question correctly.
• Answer equality is decided by a semantic-matching judge. The accepted set contains abstracted multi-hop questions, each paired with a verified, unique answer.

Usage in the Model

The accepted dataset, along with the in-house and open-source question sets, is used for trajectory generation. The paper does not specify a training split or mixture ratio in the provided text, but the dataset is designed to train a search agent on hard, solvable multi-hop questions that require composing evidence across sources.

Method

Method

The authors train a web-search agent through a fully LLM-driven data pipeline that reverse-constructs multi-hop questions from the hyperlink structure of a web corpus, followed by iterative cycles of supervised fine-tuning and reinforcement learning. The overall approach consists of two main components: a data pipeline that generates hard, solvable training questions, and a training recipe that turns those questions into reliable search behavior.

Data Pipeline

The data pipeline is organized into three stages: web-graph construction, task synthesis, and dual-criteria verification.

Web-graph construction. The authors model the corpus as a directed graph whose nodes are pages and whose edges are hyperlinks:

$$G = (V, E), \quad \operatorname{Out}(v) = \big\{u \in V \mid (v, u) \in E\big\}.$$G=(V,E),Out(v)={u∈V∣(v,u)∈E}.

Each synthesis instance begins from a seed page drawn by a sampling policy, $v_0 \sim P_{\mathrm{seed}}(V)$v0​∼Pseed​(V), for which they use an answer-anchored mode that fixes a target answer entity and retrieves pages describing it. The seed is then expanded along its out-links into a local subgraph:

$$G_{\mathrm{sub}} = \big(\{v_0\} \cup N, E_{\mathrm{sub}}\big), \qquad N = \{v_i\}_{i=1}^{k} \subseteq \operatorname{Out}(v_0),$$Gsub​=({v0​}∪N,Esub​),N={vi​}i=1k​⊆Out(v0​),

retaining the full text of each page truncated to a fixed budget. Because the reader services that render pages strip inline anchors, the authors recover the true out-link set of $v$v from a structured semantic mirror of the corpus (RDF triples) together with rendered page markup, merging the two sources to maximize link recall.

Task synthesis. Given a subgraph $G_{\mathrm{sub}}$Gsub​, this stage produces a single obscured multi-hop question through three steps: distilling the pages into an entity graph, authoring a question over that graph, and abstracting away every directly searchable anchor.

First, the authors distill $G_{\mathrm{sub}}$Gsub​ into a compact, connected entity graph:

$$G_e = (V_e, R_e) = f_{\mathrm{ext}}(G_{\mathrm{sub}}), \qquad |V_e| \leq n,$$Ge​=(Ve​,Re​)=fext​(Gsub​),∣Ve​∣≤n,

where $V_e$Ve​ are salient entities and $R_e$Re​ are typed semantic relations that preserve the cross-page link structure of $G_{\mathrm{sub}}$Gsub​. The extractor keeps only entities and relations that lie on a multi-hop path toward the seed theme, yielding a dense relational skeleton over which questions can be authored precisely.

Let $y = \mathrm{theme}(v_0)$y=theme(v0​) denote the seed theme, taken as the target answer. The authors generate an initial question together with its reasoning path over the entity graph:

$$(q_0, P) = f_{\mathrm{gen}}(G_e, y) \quad \text{s.t.} \quad |P| \geq N \land y \notin q_0,$$(q0​,P)=fgen​(Ge​,y)s.t.∣P∣≥N∧y∈/q0​,

where $P = (e_1 \xrightarrow{r_1} e_2 \xrightarrow{r_2} \cdots y)$P=(e1​r1​​e2​r2​​⋯y) is a path in $G_e$Ge​ whose traversal uniquely yields $y$y. The hard constraint $|P| \geq N$∣P∣≥N forces the question to depend on at least $N$N coupled relations.

Concrete anchors let an agent bypass the intended reasoning by directly searching a surface string. The authors remove this shortcut with an abstraction operator $\mathcal{A}$A that rewrites every non-answer entity into a descriptive reference:

$$\mathcal{A}(e): \quad \mathrm{name}(e), \mathrm{alias}(e) \notin \mathcal{A}(e) \land \mathcal{A}(e) \text{ uniquely identifies } e, \quad \forall e \in V_e \setminus \{y\}.$$A(e):name(e),alias(e)∈/A(e)∧A(e) uniquely identifies e,∀e∈Ve​∖{y}.

The initial question is then rewritten into its abstracted form:

$$\tilde{q} = f_{\mathrm{abs}}(q_0, \mathcal{A}),$$q~​=fabs​(q0​,A),

substituting each mentioned entity $e$e by $\mathcal{A}(e)$A(e) while preserving the reasoning structure and the answer $y$y. The resulting $\tilde{q}$q~​ demands disambiguation-by-reasoning rather than string matching.

Dual-criteria verification. The authors admit a pair $(\tilde{q}, y)$(q~​,y) only if it is simultaneously hard and solvable, judged by a reference model $M_{\mathrm{ref}}$Mref​ under two settings:

$$c_{\mathrm{diff}}(\tilde{q}) = \mathbb{I}\big[M_{\mathrm{ref}}(\tilde{q}) \neq y\big], \qquad c_{\mathrm{solv}}(\tilde{q}) = \mathbb{I}\big[M_{\mathrm{ref}}(\tilde{q} \mid G_e) = y\big],$$cdiff​(q~​)=I[Mref​(q~​)=y],csolv​(q~​)=I[Mref​(q~​∣Ge​)=y],

and they keep only the intersection:

$$\mathcal{D} = \big\{(\tilde{q}, y) \mid c_{\mathrm{diff}}(\tilde{q}) \cdot c_{\mathrm{solv}}(\tilde{q}) = 1\big\}.$$D={(q~​,y)∣cdiff​(q~​)⋅csolv​(q~​)=1}.

The difficulty criterion, evaluated closed-book with no tools, discards simple questions the model already answers from memory. The solvability criterion, evaluated with $G_e$Ge​ supplied as context, discards questions whose answer is wrong or non-unique. Answer equality is decided by a semantic-matching judge. The accepted set $\mathcal{D}$D of abstracted multi-hop questions, each paired with a verified unique answer, is then used for trajectory generation.

Training Recipe

The authors train the search agent through iterative cycles of supervised fine-tuning (SFT) and reinforcement learning (RL).

Supervised fine-tuning. A strong teacher $M_T$MT​ is prompted to solve each question $q \in \mathcal{D}$q∈D under the ReAct paradigm, interleaving reasoning, tool calls, and observations against live search tools. A trajectory is the resulting sequence of reasoning steps, tool calls, and observations, terminated by a final answer:

$$\tau = \left(r_1, a_1, o_1, \dots, r_T, a_T, o_T, r_{T+1}, \hat{y}\right) \sim \pi_{M_T}(\cdot \mid q, \mathcal{T}),$$τ=(r1​,a1​,o1​,…,rT​,aT​,oT​,rT+1​,y^​)∼πMT​​(⋅∣q,T),

where $r_t$rt​ is the reasoning at step $t$t, $a_t \in \mathcal{T}$at​∈T is a tool call from the tool set $\mathcal{T} = \{\mathrm{SEARCH}, \mathrm{SCRAPE}\}$T={SEARCH,SCRAPE}, $o_t$ot​ is the returned observation, and $\hat{y}$y^​ is the final answer. Each observation is a document-level summary produced on the fly rather than a raw page, keeping trajectories within a bounded context budget.

Let $\mathcal{D}_{\mathrm{raw}} = \{(q, \tau)\}$Draw​={(q,τ)} be the pool of collected trajectories. Every sample must clear a trajectory-level stage that admits $\tau$τ only if it is correct, non-degenerate, and non-trivial. Correctness is a two-part gate: the rollout must terminate successfully, and its answer must be judged correct by an LLM judge $J$J against the reference $y^*$y∗:

$$c_{\mathrm{corr}}(q, \tau) = \mathbb{I}[\mathrm{success}(\tau)] \cdot \mathbb{I}[J(q, \hat{y}_\tau, y^*) = \mathrm{CORRECT}].$$ccorr​(q,τ)=I[success(τ)]⋅I[J(q,y^​τ​,y∗)=CORRECT].

The authors then remove degenerate trajectories: repetition loops, runaway tool calling, unterminated thinking blocks, and other pathologies. Their primary detector is a sliding-window compression ratio, computed in $O(n)$O(n) over the decoded text:

$$\rho_{\mathrm{cr}}(w) = \frac{|w|}{\left|\mathrm{zlib}(w)\right|}, \quad c_{\mathrm{degen}}(\tau) = \mathbb{I}\left[\max_w \rho_{\mathrm{cr}}(w) \geq \tau_{\mathrm{cr}}\right].$$ρcr​(w)=∣zlib(w)∣∣w∣​,cdegen​(τ)=I[wmax​ρcr​(w)≥τcr​].

Repetitive text compresses far more than fluent text, so any window whose ratio reaches $\tau_{\mathrm{cr}}$τcr​ signals a loop, whatever its period and wherever it begins. Auxiliary detectors cover what compression alone can miss: periodically repeating lines, long single-character runs, and consecutive tool calls with byte-identical arguments. Finally, the authors keep only trajectories with at least $K$K tool-call turns, discarding shallow cases a direct lookup could resolve:

$$c_{\mathrm{depth}}(\tau) = \mathbb{I}[T_{\mathrm{tool}}(\tau) \geq K].$$cdepth​(τ)=I[Ttool​(τ)≥K].

An exact-duplicate pass over full message sequences then drops byte-identical trajectories while keeping distinct rollouts of the same question, giving the coarse set:

$$\mathcal{D}_{\mathrm{sft}} = \mathrm{dedup}\bigl\{(q, \tau) \in \mathcal{D}_{\mathrm{raw}} \mid c_{\mathrm{corr}} \cdot (1 - c_{\mathrm{degen}}) \cdot c_{\mathrm{depth}} = 1\bigr\}.$$Dsft​=dedup{(q,τ)∈Draw​∣ccorr​⋅(1−cdegen​)⋅cdepth​=1}.

Coarse filtering keeps or drops whole trajectories. When a question sits near the capability boundary of the teacher, however, a correct trajectory may still contain a locally poor turn: a redundant search, a hallucinated tool name, or reasoning inconsistent with the action actually taken. As a turn-level refinement, the authors label individual turns with an LLM judge. The difficulty is that a turn which looks wasteful in isolation is often a legitimate exploratory step, and because no human is in the loop, the judge has to draw that line on its own. The authors therefore induce the criteria from the data rather than hand-crafting them. Specifically, they sample some trajectories and ask the judge to critique them in free form, then consolidate the recurring failure modes into an explicit rubric used in the final judging prompt. For each assistant turn, the judge receives the question, reference answer, and a fixed local window of surrounding turns, and outputs either KEEP or MASK, represented as $m_t \in \{0, 1\}$mt​∈{0,1}. To prevent overly aggressive filtering, the authors mask at most 10% of the assistant turns in any trajectory. Masked turns remain in the context but are excluded from the training loss, providing cleaner learning signals without discarding useful interaction history.

For each assistant turn, let $C_{<t}$C<t​ denote its visible history, reconstructed by the harness's replay operator from the append-only conversation so as to stay byte-identical to what the agent conditioned on at inference. Over $\hat{\mathcal{D}}_{\mathrm{sft}}$D^sft​ the authors maximize the likelihood of the teacher's output:

$$\mathcal{L}_{\mathrm{SFT}}(\theta) = -\mathbb{E}_{(q, \tau) \sim \mathcal{D}_{\mathrm{sft}}} \sum_{t=1}^{T+1} m_t \log \pi_\theta\big(u_t \mid C_{<t}\big),$$LSFT​(θ)=−E(q,τ)∼Dsft​​t=1∑T+1​mt​logπθ​(ut​∣C<t​),

where $u_t$ut​ is the assistant output at turn $t$t, namely the reasoning and tool call $(r_t, a_t)$(rt​,at​) for $t \leq T$t≤T and the final reasoning and answer $(r_{T+1}, \hat{y})$(rT+1​,y^​) at $t = T+1$t=T+1. The per-turn mask $m_t \in \{0, 1\}$mt​∈{0,1} is supplied by fine filtering; observation, user, and system tokens carry zero loss by construction.

Reinforcement learning. The authors then optimize the agent against live search with a group-relative policy gradient. To keep long-horizon rollouts affordable without going fully asynchronous, they place the rollout regime between on- and off-policy through request-level partial rollout. To avoid depending on external APIs at training time, they co-locate an in-house model that serves as both reward judge and observation summarizer.

Long-horizon rollouts have a heavy tail: a few sessions run far longer than the rest and stall a synchronous step. Rather than discard unfinished work, the authors interrupt at the request level. Once a step has committed enough completed trajectories, in-flight over-sampled sessions are aborted and resumed at the next step from their committed prefix. Rollouts are organized as a forest whose nodes are message states, each caching its token, loss-mask, log-probability, and weight-version deltas, so a resumed trajectory is a path that splices prefixes generated under different policy weights, a mismatch corrected with truncated importance sampling. Completed turns are therefore reused rather than thrown away, which keeps rollout GPUs busy under a synchronous, co-located schedule at the cost of roughly $2\times$2× over-sampling as headroom.

Rather than call an external API, the authors run several FP8 engines of an in-house Qwen3.5-397B-A17B model inside the training cluster, alongside the actor and rollout engines. Acting as a generative reward model (GenRM), the model scores a rollout with a binary verdict on the extracted answer $\hat{y}_\tau$y^​τ​ against the reference $y^*$y∗:

$$R(q, \tau) = \mathbb{I}\left[\mathrm{GenRM}\big(q, \hat{y}_\tau, y^*\big) = \mathrm{A}\right],$$R(q,τ)=I[GenRM(q,y^​τ​,y∗)=A],

with no additive format term, since an empty or mid-thought-truncated answer extracts to $\emptyset$∅ and already scores 0 without invoking the judge. As a summarizer, the same engines compress each retrieved page into a short, query-relevant digest that supplies the observation $o_t$ot​. This is the only context-reduction mechanism in the rollout, with no message-history pruning or sliding window, so the context the policy conditions on is exactly what is trained on. Serving both roles in-cluster removes the external API dependency from the training loop, and one allocation covers training, rollout, and reward.

Iterative climbing. The authors alternate SFT and RL in iterative cycles. Each round of RL explores the current policy, after which a small set of high-quality rollouts is distilled back into the policy through supervised fine-tuning; RL then resumes from the updated model. Each such cycle is called a climb. This alternation complements the strengths of the two objectives: RL improves behaviors sampled by the current policy through relative rewards, while supervised fine-tuning directly reinforces rare but successful trajectories instead of relying on their contribution to a group-relative gradient.

For each query $q$q, the authors retain at most one rollout from the existing RL samples. Let $\bar{R}(q)$Rˉ(q) denote the pass rate within its rollout group. They select queries with $0 < \bar{R}(q) \leq 1/2$0<Rˉ(q)≤1/2, targeting cases that are solvable but not yet reliable. Among successful rollouts, they require at least $K_{\mathrm{rft}}$Krft​ tool-call turns and then select the shortest valid trajectory:

$$\mathcal{S}_q = \big\{\tau_i \mid R(q, \tau_i) = 1, T_{\mathrm{tool}}(\tau_i) \geq K_{\mathrm{rft}}\big\}, \quad \tau_q^{\star} = \underset{\tau \in \mathcal{S}_q}{\arg\min}\, T_{\mathrm{tool}}(\tau).$$Sq​={τi​∣R(q,τi​)=1,Ttool​(τi​)≥Krft​},τq⋆​=τ∈Sq​argmin​Ttool​(τ).

The depth constraint filters out lucky or trivial solutions, while the shortest-trajectory criterion discourages unnecessary search. The resulting set is deduplicated and used for SFT before the next RL round. Since the difficulty band is defined by the current policy's pass rate, it automatically shifts toward harder examples as the policy improves, providing a simple self-paced curriculum and a natural stopping signal as the candidate pool diminishes.

Experiment

The evaluation covers four benchmarks (BrowseComp, BrowseComp-ZH, DeepSearchQA, and HLE) with two model scales, Iris-mini and Iris-pro, tested under single-pass settings with and without context management. Iris-mini leads its parameter range on three benchmarks, while Iris-pro tops or ties all four, with both outperforming larger models and showing strong intrinsic search capability. Analysis of context management reveals consistent gains, especially for the smaller model and on long-horizon search tasks, though the gains are benchmark-dependent and retry strategies push scores higher at substantial inference cost. Overall, the results confirm that the training approach yields competitive search agents, with remaining gaps to frontier systems and a noted need for evaluation beyond benchmark metrics.

Iris-mini and Iris-pro achieve competitive or leading results across four search benchmarks, with Iris-mini excelling in the 30-35B range and Iris-pro leading or matching the best scores in the ~400B range. The models show strong intrinsic search capabilities, outperforming larger and heavily compute-dependent systems, though gaps remain relative to frontier models. Iris-mini outperforms XYZ-Aquila-mini by 3.4 points on BrowseComp and leads on BrowseComp-ZH and HLE in its parameter range. Iris-pro achieves the highest scores on BrowseComp, DeepSearchQA, and HLE, and ties on BrowseComp-ZH, with a 3.8-point lead over XYZ-Aquila-pro on BrowseComp. Without context management, Iris-mini and Iris-pro outperform several no-CM baselines by large margins, indicating gains are not solely from inference-time strategies. Context management consistently improves performance, with larger gains for Iris-mini than Iris-pro, as the smaller model exhausts context more often.

Context management (CM) consistently improves performance over no-CM baselines across all evaluated systems, with larger gains for the smaller Iris-mini model than for Iris-pro. The effectiveness of CM varies by benchmark, correlating with how often sessions exhaust their context, and combining discard-all with retry yields the highest scores but at substantial inference cost. CM improves all systems over their no-CM baselines, with gains up to 21.2 points for Iris-mini on BrowseComp. Iris-mini benefits more from CM than Iris-pro, as it consumes context faster and reaches limits more often. Gains are benchmark-dependent: BrowseComp benefits most, while HLE gains less despite a lower baseline, because its deficit is not context-related. On BrowseComp-ZH, three configurations (Iris-mini with retry, and Iris-pro with both CM variants) achieve the identical score of 85.1, suggesting capacity-independent limits. Combining discard-all with retry gives the strongest results in most settings, but the added inference cost makes it more an upper-bound exploration than a primary configuration.

Iris-mini and Iris-pro achieve competitive or leading results across four search benchmarks, with Iris-mini excelling in the 30-35B range and Iris-pro leading or matching the best scores in the ~400B range, outperforming larger and heavily compute-dependent systems while leaving gaps to frontier models. Context management consistently improves performance over no-CM baselines, with larger gains for Iris-mini, which exhausts context more often, and combining discard-all with retry yields the highest scores but at substantial inference cost. Gains are benchmark-dependent, with BrowseComp benefiting most and HLE less so, and some identical scores across configurations suggest capacity-independent limits.