One-sentence Summary

Researchers from Shanghai Jiao Tong University, Xi'an Jiaotong University, Peking University, East China Normal University, and National Tsing Hua University propose Wave Forcing, a generation-training-execution codesign for continuous video generation that uses chunk-wise causal attention and staggered denoising across model replicas to achieve one output chunk per forward round, and its Wave Runtime attains 147.6 DiT FPS and up to $2.53\times$2.53× the throughput of the faster of the Ulysses and Ring context-parallel baselines.

Key Contributions

• Wave Forcing (WF) is a generation, training, and execution codesign that uses chunk-wise causal attention to access earlier cleaner chunks before denoising completes and distributes staggered denoising and cache-refresh stages across full-model replicas, producing one output chunk per concurrent forward round after pipeline fill.
• A three-stage training procedure successively introduces mixed-noise denoising, motion supervision, and causal adaptation using self-generated history, adapting the model to the continuous generation process.
• Wave Runtime (WaveRT) uses Wave Parallelism (WP) for early K/V delivery and communication-computation overlap, along with optimized kernels and pipelined VAE decoding. On Wan2.1-T2V-1.3B, 5-step WF maintains overall quality comparable to Rolling Forcing while achieving higher Dynamic Degree, and 4-step WF reaches 147.6 DiT FPS and 134.8 end-to-end FPS on eight H200 GPUs; in matched DiT-only tests, WP delivers up to 2.53x throughput over the faster of the Ulysses and Ring context-parallel baselines.

Introduction

Real-time streaming video generation requires maintaining consistent appearance and natural motion while sustaining playable output, making chunk-wise autoregressive diffusion approaches such as Self Forcing and Causal Forcing important for conditioned generation. However, under the clean-history caching convention, these methods require an additional store forward pass to refresh key/value history, so sequential execution uses multiple forward passes per chunk and even one-step denoising needs two passes, limiting streaming throughput. The authors introduce Wave Forcing, a causal wavefront generation method that pipelines denoising stages and store across full-model replicas, allowing newer chunks to condition on earlier, cleaner chunks before predecessors finish and achieving one completed chunk per concurrent forward round after pipeline fill, supported by staged training and the Wave Runtime for overlapped key/value delivery and optimized execution.

Method

The authors introduce Wave Forcing (WF) to advance successive chunks before their predecessors finish denoising. WF couples a causal wavefront where newer, noisier chunks read earlier, cleaner states, but not vice versa. This is achieved through a bounded-width causal wavefront that updates cells in rounds defined by $k = i + s$k=i+s. The method retains full attention within each chunk and uses chunk-wise causal attention across video time. The attention mask $M_{\mathrm{WF}}(s, j) = \mathbf{1}[j \geq s]$MWF​(s,j)=1[j≥s] creates a lower-triangular structure over self and all cleaner predecessors, removing the reverse refinement found in Rolling Forcing.

As shown in the figure below:

To adapt the model to this causal wavefront inference, the authors employ a three-stage training process. The causal mask removes access to later, noisier active chunks, changing the learned conditioning.

Refer to the framework diagram:

Stage 1 involves mixed-noise initialization. The authors adopt a mixed-noise joint-denoising setup using bidirectional attention across chunks at different noise levels. They sample either a Rolling Forcing (RF) rollout or a Single-Frame (SF) rollout with equal probability and train the student with distribution matching distillation (DMD). Stage 2 is off-policy DMD. Following LPM 1.0, consecutive teacher chunks are re-noised at staggered timesteps, with preceding teacher chunks serving as clean K/V history. This stage retains bidirectional attention to supply stronger motion and subject guidance. The loss function combines DMD and LPIPS supervision:

Stage 3 introduces causal context and self-generated history. Each iteration uniformly samples a self-generated RF, WF, or SF rollout for on-policy DMD. WF uses the causal context including store-refreshed history from its own outputs. The branch objective is $\mathcal{L}_{3} = \frac{1}{3} \mathcal{L}_{\mathrm{RF}} + \frac{1}{3} \mathcal{L}_{\mathrm{WF}} + \frac{1}{3} \mathcal{L}_{\mathrm{SF}}$L3​=31​LRF​+31​LWF​+31​LSF​.

Wave Runtime (WaveRT) realizes Wave Parallelism (WP) by publishing K/V early from stage-owned replicas and overlapping delivery with computation. Each denoising stage and store owns a full-model GPU replica. Within a round, local projection can proceed without waiting for remote K/V, but attention must wait until the store and all legal cleaner ranks have supplied their K/V.

As shown in the figure below:

WaveRT publishes K/V after QKV projection and RoPE, then checks readiness immediately before attention. This early publication hides later waits. On a common time base, the exposed wait for a consumer is the maximum wait over required inputs. Paged delivery writes K/V directly into ordered attention slots and signals readiness before use, avoiding remote concatenation.

Local computation and K/V delivery require joint optimization. Asynchronous decoding overlaps VAE and DiT, but the slower pipeline limits end-to-end throughput. The authors use profiled streaming times to balance contiguous VAE partitions across dedicated GPUs while preserving request-local causal caches.

Refer to the framework diagram:

Decoding and inter-stage transfers must sustain the DiT output rate. WP uses full-model replicas, with execution width determined by the denoising-step count. The goal is K/V readiness before each layer's attention, reducing exposed communication and synchronization waits.

Experiment

The experiments evaluate 4-step and 5-step Wave Forcing built on Wan2.1-T2V-1.3B against Rolling Forcing and Causal Forcing using short- and long-video generation benchmarks plus subject-preservation and camera-drift diagnostics. Wave Forcing achieves the best overall balance of visual quality, motion, and long-video stability, whereas Causal Forcing shows high motion and quality but severe subject loss and camera drift, and Rolling Forcing preserves subjects but produces little motion. Training ablations show that off-policy DMD and the combined training branches are necessary for this balance. Inference and system experiments further show that WaveRT and wave-parallel execution improve throughput, with paged memory and W8A8 helping at larger scales, while VAE capacity, training complexity, fixed-conditioning evaluation, and serving latency remain open limitations.

Wave Forcing achieves the best overall balance of short-video quality, motion, and long-video stability among the compared methods. Its 5-step configuration leads VBench Total and Quality while maintaining near-perfect subject preservation and minimal camera drift. Causal Forcing reaches the highest long-video Quality and Dynamic Degree, but its low subject preservation and elevated camera drift indicate unstable long-form outputs. Wave Forcing's 5-step variant posts the top VBench Total and Quality while maintaining the highest Subject Preservation and very low Camera Drift. Causal Forcing attains the highest VBench-Long Quality and Dynamic Degree, but its much lower Subject Preservation and much higher Camera Drift mark that high motion as unusable. The 4-step Wave Forcing configuration delivers substantially higher DiT and end-to-end FPS than the rolling and causal baselines without the stability collapse seen under Causal Forcing.

The full training configuration achieves the strongest overall balance across quality, motion, subject preservation, and camera drift. Removing off-policy DMD or SF causes sharp losses in dynamic motion, while removing RF greatly increases camera drift despite high motion. The full configuration has the highest quality score and keeps camera drift relatively low. Removing off-policy DMD sharply cuts dynamic motion and reduces subject preservation. Removing RF leads to the highest dynamic motion but also a large increase in camera drift.

The evaluation compares Wave Forcing against rolling and causal baselines on short-video quality, motion, subject preservation, camera drift, and long-video stability. The 5-step Wave Forcing variant achieves the best overall balance, while Causal Forcing produces high motion and long-form quality but suffers from unstable subject and camera behavior. The 4-step Wave Forcing improves inference speed without the stability collapse seen under Causal Forcing. Ablations show that the full training configuration is necessary, as removing off-policy DMD or SF reduces dynamic motion and subject preservation, and removing RF substantially increases camera drift.