One-sentence Summary

Researchers at Nanjing University and the Institute of Automation, Chinese Academy of Sciences introduce Reframe, a post-training framework for agentic visual reasoning that aligns observation states by combining Ground-Drop GRPO with evidence-aware rewards and Observation-Gated MOPD to repair deficient tool actions mid-rollout, outperforming Mix-RL and vanilla MOPD on 14 image and video benchmarks.

Key Contributions

• The paper introduces Reframe, the first tool-use multi-teacher on-policy distillation (MOPD) framework that treats the observation state, rather than only the output distribution, as an object of alignment and unifies image and video agentic reasoning.
• To address observation neglect, Reframe combines Ground-Drop GRPO (GD-GRPO), which trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of original visual input to strengthen reliance on tool-returned observations, and Observation-Gated MOPD (OG-MOPD), which repairs deficient tool actions mid-rollout and resumes distillation from the corrected observations.
• Experiments across 14 image and video benchmarks show that Reframe outperforms Mix-RL and vanilla MOPD, surpasses dedicated experts on average, and increases observation dependence and tool gain over outcome-based GRPO. The Reframe-100K dataset, code, and model are open-sourced.

Introduction

Agentic visual reasoning equips vision-language models with image or video tools to gather fine-grained evidence in perception-heavy tasks, but tool use can suffer from observation neglect: the model invokes a tool without meaningfully using what it returns. Prior work often focuses on a single modality, while unified image-video training based on mixed-data SFT followed by RL tends to lag behind single-modality experts due to off-policy pattern conflict and sparse RL supervision. On-policy distillation methods such as OPD and MOPD provide denser feedback, but they mostly refine how supervision is routed along existing student rollouts rather than correcting the tool calls that determine which observations are available. The authors introduce Reframe, a two-stage training framework combining Ground-Drop-GRPO and Observation-Gated-MOPD. GD-GRPO enforces observation necessity through reward augmentation and stochastic dropping of the original visual input, while OG-MOPD repairs deviated tool actions before consolidation. Across 14 image and video benchmarks, Reframe improves over Mix-RL and vanilla MOPD and exhibits stronger causal reliance on tool-returned evidence.

Dataset

The authors construct Reframe-100K from heterogeneous spatial and temporal grounding annotations. It contains 100K visual QA pairs:

• 63K image QA pairs
• 37K video QA pairs

The dataset is built through a five-step cascaded data construction pipeline with several processing rules:

• Quality control: grounding annotations are audited and verified over multiple rounds so the retained evidence provides reliable supervision for visual tool use.
• Tool applicability labels: examples are labeled as beneficial, neutral, or non-beneficial based on annotation coverage. The authors avoid using base-model rollout accuracy or stronger-model judgments, which prevents model-prediction bias in the labels.
• Context-aware evidence expansion: spatial bounding boxes are scaled by 1.5 in width and height about their center and clipped to the image boundary. Temporal segments are extended by one frame at both ends. This is applied once before computing evidence coverage and constructing tool supervision.
• Unified tool metadata: spatial and temporal annotations are converted into executable tool metadata under a unified format.

In terms of usage, Reframe-100K supports both RL and OPD. The resulting annotations, tool metadata, and applicability labels are used to supervise visual tool use. Examples are not discarded only because a base model can answer them without tools, since even correctly answered questions can provide useful spatial or temporal tool-use supervision.

Method

The authors introduce Reframe, a framework designed to enhance visual tool use through a two-stage training procedure comprising Grounding-aware Decoupled Group Relative Policy Optimization (GD-GRPO) and Observation-Gated Mixture of Policy Distillation (OG-MOPD). The methodology relies on three core design choices: restricting the toolset to two fundamental vision tools (CropImage and SelectFrames), adopting Qwen3.5-9B as the base model, and utilizing a unified data corpus for both training stages.

To support this training pipeline, the authors establish a rigorous five-step cascaded data construction process to generate the Reframe-100K dataset. This pipeline prioritizes the quality of grounding annotations, data-centric tool applicability, and context-aware evidence.

As shown in the figure below:

The data construction pipeline begins by aggregating heterogeneous sources into a raw pool and removing text shortcuts that can be answered without visual input. Subsequent auditing phases evaluate the full visual input and the evidence-only input using a strong verifier to discard samples that are either too trivial or insufficiently supported by the provided evidence. Finally, a diagnosis and review stage eliminates remaining errors. Throughout this process, spatial bounding boxes are expanded by a factor of 1.5, and temporal segments are extended by one frame at both ends to preserve necessary context without introducing excessive noise. The resulting annotations are converted into a unified executable tool metadata format.

Following data construction, the authors train spatial and temporal specialists independently using GD-GRPO. This stage couples observation acquisition with observation reliance through three key mechanisms. First, the reward design evaluates trajectories based on answer correctness, tool call behavior, grounding quality, and output validity. The reward function is defined as:

where the tool call incentive $f(p)$f(p) decays with training progress $p$p according to $f(p) = \frac{\exp(-\kappa p) - \exp(-\kappa)}{1 - \exp(-\kappa)}$f(p)=1−exp(−κ)exp(−κp)−exp(−κ)​. Grounding rewards $R_{\mathrm{obs}}$Robs​ remain active throughout training, utilizing global one-to-one IoU matching for images and frame IoU for videos to encourage complementary coverage and efficient acquisition.

Second, to prevent the model from bypassing tool-returned observations, the authors implement original-observation dropout. During training, the original visual input $o_0$o0​ is stochastically dropped with a probability of 0.5 after the first successful tool call, forcing the model to rely on the acquired observations for subsequent reasoning.

Third, to prevent the dropout condition from confounding action quality evaluation, the authors compute relative advantages within drop-conditioned subgroups. For rollouts sharing the same pre-sampled dropout indicator $d_i \in \{0, 1\}$di​∈{0,1}, the advantage is calculated as:

The second training stage, OG-MOPD, addresses the issue where deficient tool actions place the student in uninformative visual states, thereby constraining teacher feedback. The authors intervene at the tool-action boundary to improve observation validity while retaining the student's generation. In the observation-gated rollouts, the student's proposed tool action $a_t$at​ is selectively replaced by the canonical action $a_t^\star$at⋆​ if it is invalid or if its IoU with the annotated evidence $e_t^\star$et⋆​ falls below a threshold $\delta$δ:

This selective intervention balances on-policy exploration with observation validity.

Finally, the authors introduce branch-aware distillation to handle the distinct token sources introduced by observation repair. The objective separates student-sampled tokens $\mathcal{S}$S and injected action tokens $\mathcal{I}$I:

where $\mathcal{L}_{\mathrm{NLL}}(\mathcal{I}) = - \frac{1}{|\mathcal{I}|} \sum_{k \in \mathcal{I}} \log \pi_\theta(y_k \mid h_k)$LNLL​(I)=−∣I∣1​∑k∈I​logπθ​(yk​∣hk​). This separation ensures that the negative log-likelihood term teaches the student how to reach informative observation states, while the standard MOPD loss transfers expert behavior along the subsequent student-sampled trajectories.

Experiment

The work evaluates Reframe on seven image and seven video benchmarks against modality-specific RL specialists, mixed and merged training baselines, and existing agentic visual reasoning models under shared visual-token budgets. Results show that Reframe outperforms all compared methods, with the proposed GD-GRPO mainly improving tool use and observation dependence rather than general visual capability, and observation repair adding further gains over specialist consolidation. Ablations indicate that dense token-level supervision matters for merging specialists, visual reasoning partially transfers across modalities, and Reframe's advantage comes largely from autonomously acquiring and using tool-returned evidence.

Reframe is the only compared model that combines unified image-video training with no SFT cold start. It achieves the highest observation dependence and the largest tool gain among the representative visual agents. This indicates that Reframe both seeks visual evidence and uses it effectively to improve reasoning. Reframe is the only model in the comparison with both unified image-video training and no SFT cold start. It reaches the highest observation dependence by a substantial margin over the next-best model. Its tool gain is clearly positive and far exceeds the best gain among the other compared models, while one alternative with relatively high observation dependence shows a negative tool gain.

Reframe achieves the strongest average results across the evaluated image and video benchmarks, improving over both the base model and vanilla MOPD. Mix-RL falls short of its modality-specific specialists, while MOPD recovers much of that gap through denser token-level supervision and can surpass specialist teachers on several benchmarks. Observation repair in Reframe contributes additional gains, with the largest improvement on VisualProbe. Mix-RL retains less of the specialist gains than MOPD and trails Image-RL and Video-RL in their respective modalities. MOPD approaches or surpasses specialist performance on several benchmarks, and Reframe adds further average gains over MOPD, particularly on VisualProbe.

Training with GD-GRPO improves an image specialist's benchmark average and observation dependence compared with outcome-based GRPO. The improvement is concentrated in tool-assisted reasoning: tool gain is larger under GD-GRPO while tool-free accuracy changes only slightly. GD-GRPO also increases accuracy under ground-truth, random, and blank observations, with the largest benefit under ground-truth observations. GD-GRPO improves average image understanding over outcome-based GRPO, while its tool-free accuracy gain remains small. Tool use provides a substantially larger accuracy gain with GD-GRPO than with outcome-based GRPO. GD-GRPO improves accuracy across ground-truth, random, and blank observations, with the largest increase under ground-truth observations. Observation dependence increases, indicating stronger use of informative observations rather than higher susceptibility to misleading ones.

Reframe achieves the best performance among the compared visual agents, leading across all evaluated benchmarks and posting the highest image and video averages. It also shows the strongest dependence on tool-returned observations and the largest gain from autonomous tool use, suggesting that its advantage comes from effective evidence acquisition and use. Controlled observations further show that its performance is strongest with ground-truth tool evidence and weaker with random or blank observations. Reframe outperforms all other listed agents on every reported benchmark, with especially large margins on image and video averages. Reframe shows the highest dependence on tool-returned observations and the largest tool gain among the compared models, indicating stronger use of acquired visual evidence.

The experiments compare visual agents on image and video benchmarks and examine training choices including unified image-video training without an SFT cold start, denser token-level supervision, observation repair, and GD-GRPO. Reframe consistently leads all evaluated benchmarks and shows the highest observation dependence and largest tool gain, indicating that it acquires and uses visual evidence more effectively than alternatives. MOPD recovers much of the specialist gap through denser token-level supervision, and observation repair adds further gains, especially on VisualProbe. GD-GRPO improves average performance and observation dependence over outcome-based GRPO, with benefits concentrated in tool-assisted reasoning and under ground-truth observations.