One-sentence Summary

National University of Singapore, Zhejiang University, and colleagues propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit that neutralizes object-centric Evidence Regions and measures the resulting support drop against non-evidence regions, integrating this signal with answer correctness within GRPO to reward evidence-intensive answers without question-specific annotations or inference overhead, and demonstrate its superior performance over prior RL post-training methods across nine benchmarks.

Key Contributions

• Counterfactual Evidence Disentanglement (CED) is introduced as a training-time audit that measures a VLM’s causal reliance on object-level evidence. It neutralizes an Evidence Region, compares the support drop against matched non-evidence regions, and uses only weak object proposals with no question-specific annotations.
• CED is integrated into GRPO as EVIDENCE-RL, a post-training framework that rewards correct answers with high evidence dependence and adds no inference-time overhead.
• Across four backbones and nine benchmarks, EVIDENCE-RL outperforms prior RL-based post-training methods; gains stem from object-centric evidence reliance, and text-only reasoning is preserved with a mean accuracy change of −0.27 pp on eight text benchmarks.

Introduction

Vision-language models (VLMs) are increasingly deployed as general-purpose visual reasoners, yet their correct-looking answers often rely on language priors or dataset shortcuts rather than on the actual visual evidence in the image. This lack of grounding persists because post-training signals that only observe text outputs cannot distinguish whether a correct answer was caused by the relevant evidence or by a non-evidence path, a problem the authors formalize as evidence-closed self-evolution. Prior perception-aware methods like PAPO and VPPO reintroduce image information but target only coarse visual dependence through global perturbations or attention proxies, not the specific local evidence that would falsify the answer if removed. The authors propose Counterfactual Evidence Disentanglement (CED), a training-time diagnostic that tests whether a model’s support for an answer drops more when a targeted evidence region is neutralized than when matched non-evidence regions are, using structured mean replacement of visual tokens. Integrated with GRPO, CED rewards correct answers that are causally grounded in the image without requiring question-specific annotations or adding inference overhead.

Method

The authors propose EVIDENCE-RL, a post-training framework designed to ground vision-language models in specific visual evidence rather than relying on language priors or irrelevant context. To formalize the problem, they decompose the causal paths leading to a candidate answer. A candidate answer can be supported by the target visual evidence, by irrelevant visual context, or by language and world priors. Standard post-training signals often fail to distinguish between these paths because they terminate at the same observed output. The qualitative cases in the figure below illustrate how the proposed mechanism redirects answers that would otherwise be driven by shortcut priors toward visual evidence.

EVIDENCE-RL addresses this ambiguity by introducing Counterfactual Evidence Disentanglement (CED), which tests whether a response specifically depends on the evidence path. The overall training pipeline is summarized in the framework diagram.

Given an image $I$I, question $q$q, and a candidate answer $y$y sampled by the policy model, the framework identifies a proposed Evidence Region $\Omega^{\mathrm{ev}}$Ωev and $K$K non-evidence Regions $\{\Omega_k^{\mathrm{non}}\}_{k=1}^K${Ωknon​}k=1K​. The Evidence Region is a weak spatial proposal resolved from dataset metadata, such as object-level annotations. The non-evidence regions serve as a sample-local null distribution, matched in area and spatially separated from the evidence region.

To measure evidence sensitivity, the authors intervene in the feature space after spatial merging. For a region $\Omega$Ω, its visual tokens are replaced with the mean $\boldsymbol{\mu}_T$μT​ of neighboring tokens:

$$\tilde{\mathbf{h}}_i = \begin{cases} \boldsymbol{\mu}_T & \text{if } i \in \mathcal{T}(\Omega), \\ \mathbf{h}_i & \text{otherwise}, \end{cases}$$h~i​={μT​hi​​if i∈T(Ω),otherwise,​

where $\mathcal{T}(\Omega)$T(Ω) maps a spatial region to visual-token indices. This feature-space mean replacement removes region-specific information while preserving the local representation manifold, reducing artifacts compared to zeroing or noise injection.

The evidence sensitivity of $y$y to region $\Omega$Ω is defined as the counterfactual log-likelihood drop:

$$s(\Omega) = \log \pi_\theta(y \mid I, q) - \log \pi_\theta(y \mid \tilde{I}_{\setminus \Omega}, q).$$s(Ω)=logπθ​(y∣I,q)−logπθ​(y∣I~∖Ω​,q).

If the model can support $y$y without inspecting $\Omega$Ω, $s(\Omega)$s(Ω) remains close to zero. To obtain a bounded and contrastive score, the framework compares the evidence-region sensitivity with the reference-region sensitivities to compute the evidence margin:

$$m(I, q, y) = \tanh \left( \frac{s(\Omega^{\mathrm{ev}}) - \mu(s^{\mathrm{non}})}{\sigma(s^{\mathrm{non}}) + \epsilon} \right),$$m(I,q,y)=tanh(σ(snon)+ϵs(Ωev)−μ(snon)​),

where $s^{\mathrm{non}} = \{s(\Omega_k^{\mathrm{non}})\}_{k=1}^K$snon={s(Ωknon​)}k=1K​, and $\mu(s^{\mathrm{non}})$μ(snon) and $\sigma(s^{\mathrm{non}})$σ(snon) are the mean and standard deviation over non-evidence regions. A positive margin indicates that the candidate answer depends more on the proposed Evidence Region than on area-matched non-evidence regions.

The causal decomposition motivating this approach is detailed in the figure below.

Finally, the authors integrate this margin into Group Relative Policy Optimization (GRPO) as a correctness-anchored reward. The training reward is defined as:

$$R_{\mathrm{train}} = R_{\mathrm{ans}} \cdot g(m) + \varepsilon_{\mathrm{tie}} m, \qquad g(m) = \frac{1}{2} \left( 1 + \tanh \left( \frac{m}{\tau_g} \right) \right),$$Rtrain​=Rans​⋅g(m)+εtie​m,g(m)=21​(1+tanh(τg​m​)),

where $\tau_g = 0.20$τg​=0.20, $\varepsilon_{\mathrm{tie}} = 0.10$εtie​=0.10, and $R_{\mathrm{ans}}$Rans​ is the answer-correctness score. When the answer is correct, $g(m)$g(m) favors responses with stronger Evidence Region support. When the answer is wrong, the evidence term acts as a bounded tiebreaker. After within-group normalization, correctness remains the dominant gradient driver, while CED ranks equally correct rollouts by their evidence support.

Experiment

Experiments confirm that the counterfactual reward provides a fine-grained evidence-dependence signal, enabling RL to improve visual grounding across nine benchmarks with no negative transfer, outperforming matched backbones and recent baselines. Removing the signal leads to net-negative deltas on general reasoning; the Answer-CED variant avoids chain-length reward hacking and generalizes across model families. Robustness tests show the reward depends on structured local interventions and remains stable under perturbations.

Counting tasks exhibit high reward variability and frequent reward conflicts, while presence tasks show low variability but strong negation effects. CoT-CED suffers from chain-length collapse on counting, whereas Answer-CED avoids this by scoring only the final answer span. On counting, 99.5% of samples produce non-constant raw reward and 90% receive different rewards for the same answer, indicating high signal sensitivity. Presence tasks show 79% zero-variance reward and a mean negation effect of 0.777, reflecting a stable but polarity-sensitive signal. CoT-CED chains shrink to a mean of 3.6 tokens on counting, degenerating into object-cue shorthand rather than grounded reasoning. Answer-CED removes the chain-length degree of freedom by scoring only the final-answer span, preventing reward-hacking via early truncation.

The proposed Answer-CED method achieves the highest average score and largest mean improvement over its Qwen2.5-VL-7B base among all compared models. It is the only RL-based method that never degrades performance, with gains concentrated on perception-heavy benchmarks like VLMsAreBlind and FREAK, while general reasoning tasks stay stable. This shows the reward successfully encourages visual evidence reliance without sacrificing broad transfer. The method delivers the highest average score and the largest mean improvement over the shared Qwen2.5-VL-7B base, surpassing both VLM backbones and RL baselines. It is the only RL method with non-negative performance change on all nine benchmarks, with the largest absolute gains on perception-heavy tasks.

Under matched data, model, and compute, only Answer-CED yields positive transfer across both grounding and general reasoning benchmarks. Correctness-only and VPPO improve grounding but hurt general reasoning, yielding net-negative average gains, while PAPO degrades severely due to repetitive loops. Answer-CED outperforms the correctness-only baseline by 12.58 points on average, with the largest improvements on evidence-sensitive tasks such as FREAK and Hallusion. Answer-CED improves all nine benchmarks over the correctness-only baseline, with the average gain reaching 12.58 points. Correctness-only and VPPO produce negative average deltas over the frozen base, as gains on grounding benchmarks are outweighed by declines on general reasoning.

The method yields non-negative gains on every benchmark for both Qwen2.5-VL backbones shown, with the largest absolute improvements on visual grounding tasks. On Qwen2.5-VL-3B, the average improvement across nine benchmarks is 4.59 points, driven by strong gains on FREAK and VLMsAreBlind while CountBench stays unchanged. On Qwen2.5-VL-3B, FREAK improves by 14.37 points and VLMsAreBlind by 7.05 points, whereas CountBench shows no change (0.00). All nine benchmark deltas for Qwen2.5-VL-3B are non-negative, resulting in an average gain of 4.59 points.

The Answer variant yields a slightly higher mean gain (+11.34) than the CoT variant (+10.60) on Qwen3.5-9B, with stronger transfer to general reasoning benchmarks such as MMMU. Although CoT achieves a larger reward during training, it suffers from reward hacking where the model shortens reasoning traces, weakening visual grounding and downstream performance; Answer avoids this by scoring only the final answer. Answer-CED outperforms CoT-CED on MMMU by over seven points (75.67 vs. 68.23), demonstrating better transfer to complex multimodal reasoning. CoT-CED lags behind Answer-CED on grounding benchmarks like SpatialEval (92.90 vs. 93.90) and general reasoning tasks like MathVista (81.88 vs. 83.75), as its chain-length manipulation undermines visual grounding instead of strengthening it.

The evaluation framework compares Answer-CED against CoT-CED and other RL-based baselines, analyzing reward signal properties and chain-length behavior across multiple vision-language models. Answer-CED scores only the final answer span, preventing CoT-CED's chain-length collapse and reward hacking on high-variance counting tasks while preserving a stable, polarity-sensitive signal on presence tasks. Across all experiments, Answer-CED is the only method that never degrades performance, achieving positive transfer on visual grounding benchmarks without harming general reasoning, and it consistently outperforms correctness-only and CoT-CED baselines on both perception-heavy and multimodal reasoning tasks.