One-sentence Summary

Researchers from KAIST and AITRICS propose LOWN, a certificate-guided visual routing decoder for diffusion vision-language models that predicts masked tokens without the image, processes the image only upon disagreement with the last image-conditioned prediction, and certifies prior predictions for reuse, retaining $99.6\%$99.6% of LLaDA-V's score on ten benchmarks at $35.3\%$35.3% decoding FLOPs without training or tuned thresholds.

Key Contributions

• A measurement study of LLaDA-V decoding shows that removing visual tokens changes the unmasked token at only 22.8% of steps on average, as few as 7.9% on MMBench, with image dependence varying across benchmarks and steps.
• LOWN is introduced as a training-free and threshold-free decoding method that decides at each step whether to process the image and how many tokens to unmask, based on agreement between a text-only forward and the last image-conditioned prediction.
• LOWN retains 99.6% of LLaDA-V's score on ten benchmarks at 35.3% of decoding FLOPs on average, and it generalizes to LLaDA-based DVLMs, composes orthogonally with visual token pruning, and runs under key-value caching.

Introduction

Diffusion vision-language models such as LLaDA-V generate responses by iteratively unmasking tokens while conditioning on visual and text inputs, but visual tokens dominate the input sequence, so standard decoding repeatedly reprocesses an unchanged image and spends most of its compute there. Prior acceleration methods use visual token pruning, intermediate-state caching, or more aggressive unmasking schedules, but they do not dynamically ask whether the current decoding step actually needs image access. Fixed schedules are also poorly matched to observed behavior because image-dependent steps vary across tasks and trajectories. The authors introduce LOWN, a training-free and threshold-free decoding method that compares a cheap text-only forward with the latest image-conditioned prediction to decide when to reprocess the image and how many tokens to unmask. LOWN retains 99.6% of LLaDA-V's score at 0.35 times the decoding FLOPs and composes with existing pruning and caching techniques.

Method

The authors propose LOWN, a decoding strategy that separates the decision of whether a response state requires visual context from the decision of how far decoding can advance. As shown in the figure below:

Instead of relying on a single FULL forward pass for both decisions, LOWN retains the last image-conditioned prediction as an anchor and tests it using a TEXT-ONLY forward pass. A FULL forward pass is executed only when the text-only test fails to certify a sufficient number of tokens. This approach alters neither the model parameters nor the attention pattern.

The method relies on two core mechanisms: the anchor certificate and certificate-guided decoding. Whenever a FULL forward pass is executed, LOWN retains its predictions and confidences on the remaining masked positions, forming the anchor. At each subsequent step, a TEXT-ONLY forward pass is performed, and the masked positions are ordered by decreasing anchor confidence. A candidate token is considered certified if its current TEXT-ONLY prediction matches the anchor prediction. Because these two predictions differ in whether they process the image and the response state from which they were computed, their agreement approximates the condition that the image is not needed for that token and that the anchor remains valid as the state evolves. The longest confidence-ordered prefix of certified candidates forms the certified set, with the acceptance length determined dynamically by the current input and step rather than a fixed constant.

In the certificate-guided decoding phase, the length of this certified prefix dictates the subsequent action. If the prefix contains at least $k$k tokens (where $k$k is the number of tokens the baseline unmask per step), the step skips the FULL forward pass entirely. It accepts the entire prefix, unmasking every certified position to its anchor prediction, which allows a single step to unmask more than $k$k tokens. Conversely, if the prefix is shorter than $k$k, the anchor no longer certifies enough tokens. The step then executes a FULL forward pass, replaces the anchor with its new predictions and confidences, and unmasks $k$k positions. These positions are selected in disagreement-first order, prioritizing positions where the TEXT-ONLY and FULL predictions differ, followed by the remaining positions ordered by FULL confidence. Both branches unmask at least $k$k tokens, ensuring LOWN never exceeds the baseline's total number of steps while potentially requiring fewer.

The authors also provide a cost analysis to quantify the computational savings. Let $\mathcal{C}(n)$C(n) denote the FLOPs of one forward pass over $n$n tokens. The relative FLOPs of LOWN compared to the baseline can be expressed as:

$$R = f + \sigma \rho$$R=f+σρ

where $f = n_F / T$f=nF​/T is the frequency of FULL forwards, $\sigma = n_T / T \leq 1$σ=nT​/T≤1 is the relative number of steps, and $\rho = \mathcal{C}(N_T) / \mathcal{C}(N_F)$ρ=C(NT​)/C(NF​) is the unit cost of a TEXT-ONLY forward relative to a FULL forward. The certificate mechanism lowers $f$f, while accepting more than $k$k tokens lowers $\sigma$σ. The input determines $\rho$ρ, which decreases as visual tokens occupy a larger portion of the FULL input. This framework can also be composed with visual token pruning, which further reduces the cost of each FULL forward pass without affecting the frequency at which they are run.

Experiment

The experiments evaluate LOWN on LLaDA-V, LaViDa, and VidLaDA across image and video benchmarks, comparing it with visual token pruning, key-value caching, and other accelerated baselines. Standalone LOWN retains nearly full task quality while substantially reducing decoding FLOPs, and it composes with pruning and caching to cut cost or latency further without retraining. Transfer results show that the method generalizes across DVLM backbones and video tasks, while ablations indicate that most savings come from selectively skipping full image processing, with adaptive acceptance adding further task-dependent reductions and certificate reliability remaining high.

At matched decoding FLOPs, LOWN preserves nearly all of the LLaDA-V baseline average quality while using roughly one third of the compute, and it outperforms standalone accelerated methods on most benchmarks. Combining LOWN with visual token pruning further reduces FLOPs and achieves the highest scores across all evaluated benchmarks, indicating that adaptive image access and token pruning are complementary. LOWN nearly matches baseline performance at a fraction of matched decoding FLOPs, whereas competitors retain lower average scores despite using more compute on average. Visual token pruning composes with LOWN to cut FLOPs further; the combined method scores highest on all ten benchmarks, and less aggressive pruning can outperform more aggressive pruning in both quality and compute.

Combining LOWN with Fast-dLLM on LLaDA-V provides substantial efficiency gains over Fast-dLLM while keeping scores close to the LLaDA-V baseline. Across the three reported benchmarks, generation time falls by roughly two-fifths to half and FLOPs fall by roughly two-fifths to three-fifths relative to Fast-dLLM. The combined scores are slightly lower than Fast-dLLM on all three tasks, with the largest decline on LLaVA-Bench. Adding LOWN to Fast-dLLM cuts generation time and FLOPs substantially, with time reduced by about two-fifths to half and FLOPs by about two-fifths to three-fifths. Fast-dLLM + LOWN remains within a small score margin of LLaDA-V but scores below Fast-dLLM on all three benchmarks, especially on LLaVA-Bench.

On VidLaDA-8B video understanding, LOWN reduces decoding FLOPs by roughly three quarters on EgoSchema and MLVU relative to the baseline. It preserves quality closely, with a small drop on EgoSchema and a slight gain on MLVU. LOWN achieves large FLOPs reductions on both video benchmarks while keeping scores close to the VidLaDA baseline. On MLVU, LOWN slightly exceeds the baseline score at a fraction of the decoding cost.

LOWN achieves a better quality-compute tradeoff than periodic and random image-access schedules on LLaDA-V. It is competitive with Period 2 while using substantially fewer FLOPs and outperforms a compute-matched random schedule across all three benchmarks. Fixed schedules such as Period 4 can match LOWN's compute on MathVista but score lower. LOWN improves task-normalized quality by 4.9-13.0% over a random schedule with the same average FULL-forward count. Period 2 achieves only a slight MathVista gain over LOWN but uses considerably higher relative FLOPs. Period 4 uses similar compute to LOWN on MathVista yet scores 4.0 points lower.

On LLaDA-V, visual routing alone lowers FLOPs by roughly 27 to 68 percent while scores remain mostly close to the baseline, with small changes across benchmarks. Adding adaptive acceptance removes a further 21 to 43 percent of remaining FLOPs and improves scores on seven of ten benchmarks, with limited declines elsewhere. Visual routing alone delivers large FLOPs reductions, roughly 27 to 68 percent across benchmarks, with only minor and mixed score changes. Adaptive acceptance further reduces remaining FLOPs by roughly 21 to 43 percent and improves or largely preserves task quality, raising scores on seven of ten benchmarks.

The experiments evaluate LOWN on LLaDA-V and VidLaDA-8B across image, video, and multimodal benchmarks, comparing it with accelerated decoding methods, Fast-dLLM, fixed and random visual access schedules, and ablations of visual routing and adaptive acceptance. LOWN consistently reduces decoding FLOPs and generation time while preserving baseline quality or slightly improving it, and it composes with visual token pruning and Fast-dLLM for further efficiency gains. On video benchmarks it cuts decoding FLOPs by roughly three quarters with only small score changes, and ablations show that routing and adaptive acceptance together yield better quality-compute tradeoffs than fixed or random schedules.