One-sentence Summary

Researchers from Peking University and collaborators propose Beacon, an agentic visual reasoning model that incorporates Necessity-Aware Adaptive Reward and Hint-Guided Capability Expansion during reinforcement learning to enable adaptive tool invocation and genuine performance gains on complex tasks, thereby overcoming the mode adaptiveness and tool effect limitations of prior multimodal large language models.

Key Contributions

• This work introduces Mode Adaptiveness and Tool Effect to systematically evaluate whether models invoke tools appropriately and whether tool use extends capabilities beyond text-only reasoning. A systematic analysis of existing models reveals limited adaptiveness and that gains on hard problems are largely offset by harm on easy ones.
• The proposed model, Beacon, integrates Necessity-Aware Adaptive Reward to encourage adaptive tool invocation based on task necessity and Hint-Guided Capability Expansion to recover learning signals from the most challenging examples during reinforcement learning.
• Extensive experiments across 13 diverse benchmarks demonstrate that Beacon achieves strong overall performance, with substantial improvements in both Mode Adaptiveness and Tool Effect, yielding genuine capability gains from tool use.

Introduction

The authors investigate agentic visual reasoning, a paradigm where multimodal large language models call external tools (such as Python code for image manipulation and numerical computation) to produce intermediate results that support final answer generation. Prior work shows empirical gains but largely overlooks whether models can adaptively decide when tools are truly needed, and whether tool use actually extends capabilities beyond tool-free reasoning. Through a systematic evaluation, the authors find that existing models suffer from limited tool-invocation adaptiveness, and the gains from tool use are often offset by introduced errors, yielding little improvement over text-only reasoning. To address this, they propose Beacon, a model trained with a reinforcement learning framework that combines a necessity-aware adaptive reward (which encourages tool use only when a problem cannot be solved without tools) and a hint-guided rollout strategy that exposes the model to effective tool-use trajectories on hard problems. Beacon achieves state-of-the-art average performance on 13 diverse visual reasoning benchmarks and demonstrates stronger tool-invocation adaptiveness along with a substantially larger net benefit from tool use.

Dataset

The authors construct a training data pool from open-source datasets, then split it into supervised fine-tuning (SFT) and reinforcement learning (RL) subsets based on model difficulty.

• Source data composition A collection of 16 benchmarks and datasets covering real-world perception, chart understanding, OCR, STEM, spatial reasoning, and real-world agentic reasoning. Most datasets have human-annotated labels. All samples that overlap with evaluation test sets are removed to prevent contamination.

• SFT data construction The base model (Qwen3-VL-8B-Instruct) is prompted five times per example. Examples answered correctly at most twice are kept as "hard" examples. For each hard example, Gemini 3.1 Pro generates a code-assisted reasoning trajectory containing Python code and execution outputs; only trajectories that produce a correct answer are retained. A refinement step with Gemini 3.1 Pro further improves trajectory quality.

• RL data construction The resulting SFT model is evaluated on the same data pool, sampling five responses per example. Examples where the model answers correctly no more than three times are selected as the RL training set. During RL training, reward signals account for both answer correctness and the effectiveness of tool use.

• Statistics and further details Dataset sizes and additional pipeline specifics are reported in Table 5 and Appendix B.

Method

The authors propose Beacon, a vision-language model that learns to autonomously decide when to invoke code execution for visual reasoning. The training follows an SFT-then-RL paradigm. First, the model is equipped with basic code-use skills through supervised fine-tuning on synthesized trajectories. Then, reinforcement learning with a necessity-aware reward and a hint-guided rollout mechanism further refines the model’s adaptive reasoning and the genuine benefit derived from tool use.

Training Data Construction. The data pool is built from 16 diverse benchmarks covering perception, chart understanding, OCR, STEM, spatial reasoning, and agentic tasks. For SFT, the base model (Qwen3-VL-8B-Instruct) is evaluated on the source data, and only challenging examples (solved correctly at most twice in five attempts) are retained. An expert model (Gemini 3.1 Pro) generates code-assisted reasoning trajectories for these hard examples, and the trajectories are further refined by the same expert to improve quality. The resulting trajectories form the SFT dataset. For RL, the SFT model is evaluated on the same pool, and the remaining unsolved examples (correct at most three times in five attempts) are selected as the RL training set.

Cold-Start SFT. Standard cross-entropy loss is used, with the code output (content between <tool_call> and </tool_call>) masked. To prevent the model from becoming biased toward code use and to mitigate forgetting, a small number of correct pure-text trajectories generated by the base model are injected into the SFT data.

Reinforcement Learning. The RL stage uses GRPO as the base algorithm and introduces two key components.

Necessity-Aware Adaptive Reward (NAAR). A central challenge is to discourage unnecessary tool use without harming the model’s ability to solve difficult problems with code. NAAR assigns a soft preference for text-only reasoning. For each rollout group, if at least one correct text-only response exists, correct text-only responses receive the full reward (1), while correct code-based responses receive a reduced reward (0.25). If no text-only response is correct, correct code-based responses receive the full reward. This design encourages the model to avoid tool calls when text reasoning suffices, while still rewarding successful code-based solutions when text alone fails.

Hint-Guided Capability Expansion (HCE). Hard examples that the policy cannot solve within a limited number of rollouts provide little learning signal. HCE rescues such examples by injecting expert-generated hints into the rollout prompt. For each problem, a group of responses is first sampled from the original prompt. If none is correct, a strong expert model (Gemini 3.1 Pro) produces a verified code-assisted reasoning trajectory and extracts an answer‑free hint containing only intermediate instructions, expected subgoals, and tool‑use strategies. The hint is appended to the original question, and the policy samples another group of responses under the hinted prompt. During policy optimization, the hint is removed from the model input while the trajectories generated with the hint are retained, thereby transferring the capability to the hint‑free policy.

Policy Optimization. The reward for each response combines a format reward (ensuring each code block is followed by an observation and the final answer is placed in <answer>...</answer>) and the adaptive reward:

$$R(y_i; \mathcal{G}) = 0.1 R_{\text{format}}(y_i) + 0.9 R_{\text{adaptive}}(y_i; \mathcal{G}).$$R(yi​;G)=0.1Rformat​(yi​)+0.9Radaptive​(yi​;G).

For each group, group-relative advantages are computed. The actor loss uses the clipped GRPO objective. For normal groups, the importance sampling ratio is computed with the original prompt; for hinted groups, the ratio uses the original prompt for the current policy while keeping the hinted prompt in the old policy’s context, ensuring off‑policy stability. Groups that provide no accuracy advantage (all responses correct or wrong) or no adaptive advantage (all responses share the same reasoning mode) are filtered out during training.

Experiment

The experiments first evaluate the tool-use behavior of existing agentic visual reasoning models, revealing that they show limited mode adaptiveness and little net benefit from tool use, often failing to adapt their reasoning mode to problem difficulty. Subsequently, Beacon is assessed on a diverse set of challenging benchmarks, where it consistently outperforms open-source baselines and achieves a 6.07-point average improvement over its base model. Beacon's design, incorporating a necessity-aware adaptive reward and hint-guided capability expansion, leads to the strongest mode adaptiveness and a clear positive gap between tool-induced gains and harms, demonstrating that it can effectively leverage tools for hard problems without compromising performance on easier ones.

On high-resolution visual search and spatial reasoning benchmarks, the closed-source Gemini 3.1 pro achieves the highest overall average, with a particularly large lead on the difficult BabyVision perceptual task. Among open-source models, Beacon-RL-8B delivers the best average performance, ranking first on 11 of 13 benchmarks and improving over its base model Qwen3-VL-8B-Instruct by 6.07 points. While the base model already shows strong visual search results, all open-source models struggle with BabyVision, where scores remain below 15%. Gemini 3.1 pro leads all models with an average of 73.66, and its BabyVision score of 48.45 far exceeds any open-source competitor. Beacon-RL-8B improves over Qwen3-VL-8B-Instruct by 6.07 average points, securing the top open-source rank on 11 of 13 benchmarks. Qwen3-VL-8B-Instruct achieves the best high-resolution visual search scores among the listed open-source models, with 84.85 on V* and 78.13 on HR-Bench 4K. BabyVision is the most challenging benchmark: all open-source models score below 15%, while Gemini 3.1 pro reaches 48.45.

Open-source model Beacon-RL-8B leads all open-source entries with an average score of 50.04, improving over its Qwen3-VL-8B-Instruct base by 6.07 points and ranking first on 11 of 13 benchmarks. The closed-source Gemini 3.1 Pro far exceeds all models with an average of 76.23, while among other open-source models, Qwen3-VL-8B-Instruct (43.97) outperforms Pixel-Reasoner-7B (38.11) and Thyme-7B (33.76). Beacon-RL-8B achieves the highest open-source average (50.04), surpassing its base model Qwen3-VL-8B-Instruct (43.97) by 6.07 points. Gemini 3.1 Pro leads with an average of 76.23, with particularly large gaps in GameQA (81.00 vs. 36.70 for Qwen3-VL-8B-Instruct) and TIR-Bench (47.57 vs. 19.01), showing the difficulty of compositional and agentic reasoning for open-source models.

Beacon demonstrates the strongest mode adaptiveness and tool effect among evaluated models. It consistently improves its tool-free reasoning accuracy when tools are available, achieving the largest average accuracy gain (+1.96%) and the highest positive gap between tool-induced gains and harms (Δ_TE = +3.14%). In contrast, other models exhibit only marginal tool benefits and near-zero tool effect differences. Beacon achieves a 100% Text-Retain on HRBench4K, meaning all problems solved by text-only reasoning remain correct when tools are available, while other models show lower retention. Beacon's tool effect advantage (Δ_TE) reaches +5.51 on HRBench4K and averages +3.14% overall, far exceeding the near-zero values of competing models, indicating effective tool use with minimal harm.

The full Beacon method combining necessity-aware adaptive reward and hint-guided capability expansion achieves the highest overall accuracy. Using the adaptive reward alone yields the best mode adaptiveness, while adding hint-guided capability expansion consistently improves the tool effect metric over the GRPO baseline. The combination produces the largest positive gap between tool-induced gains and harms. The necessity-aware adaptive reward alone produces the highest mode adaptiveness score, substantially above the GRPO-only baseline. Adding hint-guided capability expansion to GRPO raises the tool effect metric, indicating more effective tool use. The full method with both components achieves the best overall accuracy averaged across benchmarks and the largest positive tool effect gap.

The evaluation spans high-resolution visual search, spatial reasoning, and compositional agentic benchmarks, comparing closed-source Gemini 3.1 Pro with open-source vision-language models. Gemini 3.1 Pro dominates overall, particularly on the difficult BabyVision perceptual task, while among open models, Beacon-RL-8B achieves the highest average and improves over its base Qwen3-VL-8B-Instruct. Beacon further demonstrates the strongest mode adaptiveness and tool use, with the full method combining necessity-aware adaptive reward and hint-guided capability expansion yielding the best overall accuracy and the largest positive gap between tool-induced gains and harms.