Command Palette
Search for a command to run...
스파이크 속에서의 재고: 단계 인지 디코딩을 통한 다중모드 확산 언어 모델의 환각 완화
스파이크 속에서의 재고: 단계 인지 디코딩을 통한 다중모드 확산 언어 모델의 환각 완화
초록
다중모드 확산 언어 모델(MDLM)의 최근 발전은 반복적 정제를 통해 강력한 전역 모델링 능력을 보여주었다. 우리는 반복적 디노이징 과정에서 마스크 불확실성이 전반적으로 감소하는 추세를 보이는 반면, 일부 디노이징 단계에서는 환각과 밀접하게 연관된 국소적 엔트로피 스파이크가 나타남을 관찰하였다. 우리는 각 디노이징 단계의 엔트로피 편차를 인접한 시간 윈도우와 비교하여 측정함으로써 이상 단계를 식별할 수 있다고 주장한다. 또한 이산 토큰 확정이 마스크 위치의 분포 정보를 폐기함으로써 엔트로피 스파이크 동안 후보 의미와 문맥 간의 지속적인 상호작용을 제한한다고 가설을 세운다. 이러한 목표를 바탕으로 우리는 의미적 문맥을 활용하여 신뢰성 있는 디노이징을 달성하는 플러그 앤 플레이 방식의 디코딩 전략인 StepRefine을 제시한다. 본 방법의 핵심은 이산 디노이징과 잠재 정제 간의 단계 인지적 전환에 있다. 모델은 엔트로피 스파이크 단계에서 확률 가중 임베딩을 사용하여 다양한 추론 가설을 보존하고 여러 차례의 연속적 정제를 수행한다. 동시에 정제 컨트롤러가 분포 안정성을 모니터링하여 불필요한 반복을 줄이고 모델을 표준 이산 디노이징으로 다시 전환한다. 광범위한 실험을 통해 StepRefine은 다중모드 벤치마크에서 다양한 MDLM에 걸쳐 유의미한 환각 완화 효과를 입증하였다.
One-sentence Summary
Researchers from Monash University, Cornell University, UC Santa Cruz, Northeastern University, and the University of Minnesota propose StepRefine, a plug-and-play decoding strategy for multimodal diffusion language models that uses step-aware switching between discrete denoising and probability-weighted latent refinement to preserve diverse reasoning hypotheses, while a refinement controller monitors distributional stability to reduce unnecessary loops and mitigates hallucinations across multimodal benchmarks.
Key Contributions
- The paper identifies that mask uncertainty in multimodal diffusion language models generally decreases during denoising, but localized entropy spikes at specific steps are closely associated with hallucinated tokens.
- StepRefine is introduced as a plug-and-play, training-free decoding strategy that detects entropy spike steps via local trend deviation, switches to probability-weighted embedding-based latent refinement to preserve diverse reasoning hypotheses, and uses a refinement controller to monitor distributional stability and reduce unnecessary refinement loops.
- Experiments across four multimodal diffusion language model backbones and multiple benchmarks covering general reasoning, hallucination, mathematical, and scientific reasoning show that StepRefine improves accuracy, significantly mitigates hallucination, and can be combined with existing acceleration methods.
Introduction
Diffusion language models generate text through iterative denoising instead of left-to-right decoding, and multimodal variants called MDLMs use bidirectional attention over visual and textual inputs to support perception and reasoning. However, MDLMs still produce hallucinations, and existing decoding strategies mostly optimize inference efficiency through cache reuse or scheduling, while quality-oriented methods adjust token commitment but treat uncertainty uniformly across denoising steps. The authors analyze step-wise uncertainty dynamics and find that rare entropy spike steps, where masked-sequence entropy sharply increases locally, are strongly associated with hallucinated final tokens. They propose StepRefine, a plug-and-play decoding method that detects these entropy spikes from local temporal trends and switches to latent probabilistic refinement before discrete decoding, improving reliability across multiple MDLM backbones and reasoning benchmarks.
Method
The authors propose StepRefine, a step-aware decoding strategy designed to mitigate hallucinations in Masked Discrete Language Models. The framework operates by detecting specific instability points during the iterative denoising process and applying targeted refinement to stabilize token commitments.
The first core module is entropy spike detection. During iterative denoising, the mean mask entropy over masked positions generally decreases as the model gradually commits tokens. However, this decrease is not strictly monotonic. A small number of steps exhibit sharp local increases that break the overall trend, which the authors identify as entropy spikes. The mean mask entropy at each denoising step t is defined as the expected Shannon entropy of the predictive distribution over masked positions:
Hˉt=Ei∼U(Mt)[−v∈V∑pti(v)logpti(v)]where U(Mt) denotes the uniform distribution over the masked positions Mt. A step is classified as an entropy spike if its entropy change exceeds a local upper bound:
ΔHˉt≜Hˉt−Hˉt+1,Tspike≜{t∣ΔHˉt>μt+σt}where μt and σt represent the mean and standard deviation of the w preceding stepwise entropy changes along the denoising trajectory.
Refer to the figure below for a detailed characterization of these entropy spikes across decoding steps.
The analysis reveals that while entropy spike steps are relatively rare, approximately 80% of hallucinated tokens are located within these spike regions. Furthermore, these spikes are non-uniformly distributed, concentrating mainly in the early to middle stages of the denoising process when many positions remain masked and the available context is insufficient. At these specific steps, confidence signals become less discriminative, making discrete commitment decisions less reliably grounded in the current contextual information and rendering standard low confidence remasking less effective.
To address this vulnerability, the authors introduce a step-aware mode switch with adaptive termination. This strategy alternates between standard discrete decoding and latent refinement, gated by the previously defined spike detection criterion. At each denoising step t, the model first performs a standard forward pass to obtain the predictive distribution and the corresponding spike indicator. For ordinary steps where t∈/Tspike, the process follows the standard discrete decoding path.
Conversely, at entropy spike steps where t∈Tspike, StepRefine activates latent refinement. This involves performing iterative forward passes within the same denoising step before discrete decoding. Let p~t,0i≜pti denote the initial distribution from the standard forward pass. At each refinement iteration k≥1, the framework constructs a probability-weighted embedding from the previous distribution and replaces the mask embedding as input to the next forward pass:
e~t,ki=Ev∼p~t,k−1i[e(v)],i∈Mt,k=1,2,…Together with the forward pass, this equation forms a recursive refinement loop. It carries distributional information about candidate tokens into subsequent forward passes, allowing the soft embedding to interact with the visual-textual context prior to discrete decoding.
To prevent unnecessary computational overhead, a refinement controller monitors the change between consecutive predictive distributions. At iteration k≥1, the change is computed as the average Kullback-Leibler divergence at masked positions:
DKL(k)=Ei∼U(Mt)[DKL(p~t,ki∥p~t,k−1i)]The iterative refinement terminates once DKL(k)<ϵ, where ϵ>0 is a predefined stability threshold. The final refined distribution is then utilized for standard discrete decoding and remasking to commit tokens. To maintain a fixed inference budget, the additional forward passes required for refinement are counted against the total denoising budget T. If k refinement passes are utilized at step t, the subsequent iteration begins at index t−1−k rather than t−1, ensuring the total number of forward passes remains constant.
Experiment
The authors evaluate StepRefine on four multimodal diffusion language models across diverse reasoning, hallucination, math, and science benchmarks. Main results show that entropy-spike-aware refinement improves over baseline and existing training-free decoding strategies across backbones, while preserving speedups from acceleration methods. Qualitative analysis and ablations confirm that refinement sharpens predictions at unstable steps, and that moderate calibration windows with adaptive scheduling and termination are more effective than fixed alternatives. The method also remains robust across remasking strategies, generation lengths, and step budgets.
StepRefine consistently improves benchmark performance over the baseline and competing training-free decoding strategies across multiple MDLM backbones. It matches or outperforms CoTA, ReMix, and WINO on general reasoning benchmarks and also yields positive gains on hallucination benchmarks. The similar average improvements across backbones suggest that the entropy-spike-aware refinement is robust across MDLMs rather than specific to any single model. StepRefine improves average general reasoning accuracy over the baseline and matches or outperforms CoTA, ReMix, and WINO on every evaluated general reasoning benchmark. Hallucination benchmarks also benefit, with gains on MMHalu and Bingo, and every backbone shows positive average improvements ranging from 2.3 to 2.8 percentage points.
StepRefine integrates with existing diffusion language model acceleration methods while preserving their speedups. Across the reported accelerators, adding StepRefine improves benchmark accuracy and hallucination scores, with throughput preserved or slightly higher. The results show that refinement is broadly compatible with aggressive and cache-based decoding strategies. Adding StepRefine to Fast-dLLM improves VStar and MMVP accuracy and hallucination scores while keeping speedup near its accelerated baseline and throughput slightly higher. For dLLM-Cache, StepRefine improves all reported accuracy and score metrics, with speedup and throughput nearly unchanged. Across acceleration methods, StepRefine maintains or slightly increases throughput and shows gains on VStar and MMHalu, supporting compatibility without hidden latency.
StepRefine consistently improved MMHalu over the baseline under all three evaluated remasking strategies, with the Low Confidence setting showing the largest StepRefine score. Results across two generation lengths and three step budgets also show consistent gains, supporting robustness to decoding settings without per-setting tuning. StepRefine outperformed the baseline under Random, Low Confidence, and Top-k Margin remasking strategies. Low Confidence remasking produced the highest StepRefine score among the reported strategy rows. Reported MMHalu gains across generation lengths and step budgets were positive in all six configurations, ranging from +1.8% to +4.8%.
StepRefine is evaluated across multiple masked diffusion language model backbones, where it consistently improves general reasoning and hallucination benchmarks over the baseline and competing training-free decoding strategies, indicating robustness beyond any single model. Additional experiments show that StepRefine integrates with existing acceleration and cache-based decoding methods, preserving or slightly improving throughput while retaining accuracy gains. Ablations over remasking strategies, generation lengths, and step budgets further confirm consistent improvements without per-setting tuning.