Command Palette
Search for a command to run...
ZEUS : accélérer les modèles de diffusion à l’aide d’un prédicteur de second ordre uniquement
ZEUS : accélérer les modèles de diffusion à l’aide d’un prédicteur de second ordre uniquement
Yixiao Wang Ting Jiang Zishan Shao Hancheng Ye Jingwei Sun Mingyuan Ma Jianyi Zhang Yiran Chen Hai Li
Résumé
Les modèles génératifs par débruitage produisent des synthèses de haute fidélité, mais restent limités par la latence d’inférence due aux nombreux appels itératifs au débruitage nécessaires pendant l’échantillonnage. Les méthodes d’accélération sans entraînement réduisent cette latence soit en allégeant l’architecture du modèle, soit en raccourcissant la trajectoire d’échantillonnage. Les méthodes actuelles sans entraînement sont plus complexes que nécessaire : les prédicteurs d’ordre supérieur amplifient l’erreur lors d’accélérations agressives, et les modifications architecturales entravent le déploiement. Au-delà d’une accélération de 2×, le saut d’étapes crée une rareté structurelle — au plus une évaluation nouvelle par fenêtre locale —, ce qui fait de la sortie calculée et de sa différence rétrograde les seules informations causalement fondées. Sur cette base, nous proposons ZEUS, une méthode d’accélération qui prédit des évaluations réduites du débruitage à l’aide d’un prédicteur de second ordre et stabilise les sauts consécutifs agressifs grâce à un schéma entrelacé évitant les extrapolations successives. ZEUS n’ajoute pratiquement aucun surcoût, aucun cache de caractéristiques ni aucune modification architecturale, et il est compatible avec différents squelettes de réseau, objectifs de prédiction et choix de solveurs. En génération d’images comme de vidéos, ZEUS améliore systématiquement le compromis vitesse–fidélité par rapport aux références récentes sans entraînement, atteignant jusqu’à 3,2× d’accélération de bout en bout tout en préservant la qualité perceptuelle. Notre code est disponible à l’adresse : https://github.com/Ting-Justin-Jiang/ZEUS.
One-sentence Summary
The authors propose ZEUS, a training-free diffusion acceleration method that predicts reduced denoiser evaluations using only a second-order predictor and an interleaved scheme that avoids back-to-back extrapolations, adding essentially zero overhead without feature caches or architectural modifications while achieving up to 3.2× end-to-end speedup in image and video generation with preserved perceptual quality.
Key Contributions
- ZEUS is a training-free sampling acceleration method that reduces denoiser evaluations using a second-order predictor and stabilizes aggressive consecutive skipping with an interleaved scheme that avoids back-to-back extrapolations.
- ZEUS adds essentially zero overhead, requires no feature caches or architectural modifications, and is compatible with different backbones, prediction objectives, and solver choices because it only changes the sampling schedule.
- Across image and video generation, ZEUS consistently improves the speed-fidelity tradeoff over recent training-free baselines, achieving up to 3.2× end-to-end speedup on Flux while improving LPIPS, FID, and PSNR.
Introduction
Denoising generative models sample by integrating a probability-flow ODE, and each integration step requires a full pass through a large denoiser network, which creates substantial latency for large image and video models. Existing training-free acceleration methods reduce denoiser calls by sparsifying attention or forecasting skipped steps, but aggressive end-to-end speedups beyond 2x often degrade fidelity; higher-order trajectory predictors become unreliable when fresh evaluations are scarce, and many methods impose deployment burdens such as custom kernels or large feature caches. The authors propose ZEUS, a minimal training-free method that uses a second-order predictor based on only the most recent denoiser output and its backward difference, combined with an interleaved reuse schedule to stay stable over long skipped intervals. ZEUS requires no architecture changes or intermediate feature caches and improves speed-fidelity trade-offs across multiple image and video backbones, achieving up to 3.22x speedup on images and 2.24x on video while preserving sample fidelity.
Method
The authors propose ZEUS, a training-free acceleration framework for ODE-based generative models that accelerates discretized probability-flow ODE (PF-ODE) sampling through trajectory step skipping. Refer to the framework diagram for an overview of the system design and its integration with various backbones.
The method operates on a uniform interleaved 1:r schedule. Within each block of r+1 solver steps, the framework performs one full denoiser evaluation and skips the remaining r evaluations. To execute these reduced steps, the solver still requires a denoiser output at each step; therefore, the authors must provide an approximation ψ^t using a numerical predictor built from the trajectory information available at the full-evaluation steps.
Under aggressive scarcity where r≥2, the authors demonstrate that higher numerical order does not imply higher accuracy. Instead of relying on distant evaluations, they identify the "observed information set" at a full evaluation step t. This set consists of the fresh network output ψt and the backward difference Δ(1)ψt:=ψt−ψ^t+1. This pair constitutes the complete causal record of the denoising trajectory at step t, capturing the response of the denoiser to the given approximated state.
Using this observed information set, the authors derive a second-order predictor:
ψ^t−1=ψt+Δ(1)ψt=2ψt−ψ^t+1This predictor uses exactly the two available signals and serves as the best linear unbiased estimator under scarcity. It avoids the Ω(Δ) bias floor of reuse-only methods and the variance explosion of higher-order predictors. As shown in the figure below, the second-order predictor consistently outperforms first- and third-order alternatives in both LPIPS and MSE metrics.
When approximating multiple consecutive reduced steps (r≥2), the choice of approximation strategy significantly affects stability. Simply chaining the second-order rule (Predictor-only) amplifies errors linearly and causes the approximations to drift and overshoot. Conversely, reusing the same output for all steps (Reuse-only) avoids error amplification but ignores slope information, causing the approximations to lag and degrading fine details.
The authors resolve this tradeoff by introducing an interleaved approximation scheme. Rather than chaining predictions, the framework reuses the observed information set across all r reduced steps by cycling between the full evaluation and its second-order extrapolation:
ψ^t−j={2ψt−ψ^t+1ψtj oddj evenThis pattern applies the second-order predictor once at the most critical step (j=1) and resets to ψt for even steps. Because no new extrapolation occurs beyond j=1, variance remains bounded at O(σ2) while second-order accuracy is preserved where it matters most. As shown in the figure below, this interleaved reuse of the observed information pair prevents overshoot and preserves perceptual quality compared to the baseline schemes.
Experiment
The experiments evaluate ZEUS on five text-conditioned generative models, including text-to-image backbones such as Stable Diffusion 2.1, SDXL, and Flux.1-dev, and text-to-video backbones such as Wan2.1 and CogVideoX, using perceptual similarity, distributional drift, and wall-clock speedup metrics. Main results show that ZEUS consistently shifts the speed-quality frontier outward across all models and solvers, achieving state-of-the-art or near-state-of-the-art efficiency on image generation and a new quality-efficiency frontier on video generation without video-specific modifications. Ablations further validate that ZEUS remains effective under aggressive few-step sampling and across different solver orders, where higher-order solvers improve perceptual fidelity while speedup and PSNR stay largely unchanged.
This ablation compares Lagrange extrapolation predictors of different orders with ZEUS under varying acceleration ratios. Higher-order Lagrange prediction does not improve accuracy: third-order is consistently worse than second-order, and error grows as acceleration becomes more aggressive. ZEUS, which uses local solver information rather than stale full evaluations, achieves the best or near-best error at higher acceleration ratios while providing increasing speedups. Third-order Lagrange predictors consistently underperform second-order predictors across all tested acceleration ratios, despite access to an additional full evaluation. Prediction error increases with more consecutive reduced steps regardless of predictor order, showing that stale full denoiser evaluations become less reliable under aggressive acceleration. ZEUS achieves lower error than both Lagrange baselines at moderate and aggressive acceleration ratios, while second-order Lagrange is slightly more accurate at the mildest setting.
ZEUS shifts the speed-quality frontier outward across Stable Diffusion backbones and Flux under multiple solvers. In the SD-2 comparison, ZEUS-Medium achieves the highest DPM++ speedup while staying close to the best perceptual and distributional metrics. The broader results indicate ZEUS improves efficiency without sacrificing perceptual fidelity. ZEUS-Medium records the highest DPM++ speedup among reported SD-2 methods while remaining close to the strongest LPIPS and FID scores. Across backbones and solvers, ZEUS consistently improves the speed-quality trade-off rather than relying on a specific solver choice. On Flux, quality-leaning ZEUS presets remain on the perceptual frontier for LPIPS and FID even when other methods are competitive on runtime.
The comparison shows that chaining the second-order predictor gives second-order bias but variance that grows quadratically with consecutive steps, making it precise yet unstable. Reusing the full evaluation keeps variance at the noise level but only achieves first-order bias, so it is stable but imprecise. ZEUS matches second-order bias at the first step and otherwise follows the stable first-order bias while bounding variance within a constant factor of the noise variance. Predictor-only suffers from variance that grows quadratically with the number of consecutive steps, which explains its progressive drift. ZEUS achieves bounded variance across consecutive reduced steps while retaining second-order bias only at the first step, balancing precision and stability.
On Wan 2.1, the balanced method achieves the best perceptual similarity and highest PSNR while also improving throughput over SVG and SVG2. The fast variant reaches the highest speedup and improves quality over SVG2-Turbo, indicating a favorable speed-quality trade-off. The balanced method attains the lowest LPIPS and highest PSNR, with throughput above SVG and SVG2 while SSIM remains slightly below SVG2. The fast variant achieves the largest speedup and shows better PSNR, SSIM, and LPIPS than SVG2-Turbo, keeping quality competitive with the strongest baselines.
Few-step sampling across Flux and SDXL shows that reducing step budgets improves perceptual similarity, with LPIPS decreasing, while distributional drift measured by FID remains modest. The same qualitative trend holds across Flux and SDXL backbones and across scheduler choices, indicating the benefit is not tied to a specific backbone or solver. With fewer sampling steps, LPIPS improves for Flux and SDXL, while FID increases only modestly. The few-step trend persists across Flux and SDXL backbones and across scheduler choices, supporting robustness.
The evaluation spans predictor ablations, speed-quality comparisons, bias-variance analysis, video generation, and few-step sampling. In the ablation, higher-order Lagrange extrapolation does not improve accuracy and degrades under aggressive acceleration, while ZEUS achieves the best or near-best error by relying on local solver information. Bias-variance experiments attribute this to ZEUS keeping second-order bias only at the first reduced step and otherwise following a stable first-order bias with bounded variance. Across Stable Diffusion, Flux, and Wan 2.1, ZEUS shifts the speed-quality frontier outward, with the Wan balanced variant improving perceptual quality and throughput and the fast variant offering the largest speedup; few-step sampling also improves perceptual similarity with only modest distributional drift across backbones and schedulers.