One-sentence Summary

Researchers from Zhejiang University and Alibaba Group propose InfinityEdit, a lightweight edit-ignition adapter that equips a streaming video generator with unbounded editing ability for infinite video editing; it uses history cross-attention, temporal causal self-attention, and edit cross-attention, activated only in edit chunks, to faithfully continue streams while preserving generation stability over unbounded edit sequences.

Key Contributions

• The paper formalizes the infinite video editing task, where generated segments must continue an ongoing stream under sequential edit requests, and introduces a data-collection pipeline for this setting.
• The paper presents InfinityEdit, a lightweight edit adapter for a frozen streaming video generator such as Helios-Distilled. It combines history cross-attention, temporal causal self-attention, and edit cross-attention, and at inference it is activated only in chunks where an edit request arrives while later chunks use the original generator with a reset anchor frame.
• Extensive experiments show that InfinityEdit outperforms baseline methods by following edit requests more faithfully, continuing the stream rather than performing frame-wise rewriting, and remaining stable as edits accumulate over long sequences.

Introduction

Existing diffusion-based video editors are designed for fixed-length clips, where the output has the same duration and temporal alignment as the source. This in-place editing paradigm breaks down for open-ended streams such as live game restyling or continuous camera moves, where new content arrives indefinitely and edits must be applied to future frames. Streaming video generators can continue a clip but are not trained to follow relational edit instructions, and their stability mechanisms often resist the intended change. The authors introduce infinite video editing, a task where an edit request modifies an ongoing stream by generating the next segment as a faithful continuation. They propose InfinityEdit, a lightweight adapter that equips a frozen streaming generator with three attention modules (history cross-attention, temporal causal self-attention, and edit cross-attention) and uses an ignite-then-continue inference strategy with anchored sliding-window history to maintain stability across repeated edits.

Dataset

The authors construct a dataset of video editing triplets, each consisting of a preceding video $V_{\mathrm{pcd}}$Vpcd​, an edit instruction $c_{\mathrm{edit}}$cedit​, and a target video $V_{\mathrm{tgt}}$Vtgt​ that continues the preceding clip while applying the instruction. The dataset is built through a multi-stage pipeline and is used to train the model described in the paper.

• Sources and composition

  • Source videos $V_{\mathrm{pcd}}$Vpcd​ are sampled from the UltraVideo dataset and resized to a fixed resolution and frame count.
  • Basic edit types are drawn from representative instruction sets used in VAP; these abstract phrases are then expanded into concrete edit instructions $c_{\mathrm{edit}}$cedit​ by Gemini 3 Flash, using the captions of the source videos to ground the instruction in specific entities and scenes.
  • Target videos $V_{\mathrm{tgt}}$Vtgt​ are synthesized with the image-to-video model Wan2.2-I2V-A14B. For appearance edits (e.g., style changes), the last frame of $V_{\mathrm{pcd}}$Vpcd​ is first edited by Qwen-Image-Edit-2511 to reflect the desired change, and that modified frame serves as the starting frame for generation. For camera-motion edits, the original last frame is used unchanged.
  • The resulting triplets form the full dataset; the paper does not report the total number of samples.

• Post-processing and filtering

  • All videos are resized to match the default training resolution of the base model (Helios-Distilled).
  • Frame counts are clipped so that the ratio $T_1 / T_2$T1​/T2​ aligns with the model’s input and output window lengths.
  • Triplets are manually scored by 20 human labelers on four criteria, each rated from 1 (worst) to 4 (best):
    • Alignment – how well $V_{\mathrm{tgt}}$Vtgt​ fulfills $c_{\mathrm{edit}}$cedit​.
    • Consistency – how well $V_{\mathrm{tgt}}$Vtgt​ continues from $V_{\mathrm{pcd}}$Vpcd​ (subject stability, scene agreement).
    • Rationality – absence of implausible artifacts.
    • Visual quality – quality of the selected source clip.
  • Triplets are retained based on these scores (exact thresholds not specified).

• Usage in the model

  • The dataset is used for training the editing model, with Helios-Distilled serving as the backbone. The paper does not mention a separate validation or test split; all filtered triplets are employed in training after the format adjustments described above.

Method

To train the model for infinite video editing, the authors construct triplets consisting of a preceding video $V_{\mathrm{pcd}}$Vpcd​, an edit instruction $c_{\mathrm{edit}}$cedit​, and a target video $V_{\mathrm{tgt}}$Vtgt​. Source videos are sampled and resized, while basic edit types are expanded into detailed instructions using a vision-language model. Target videos are synthesized using an image-to-video model. For appearance edits, the connection frame is edited first; for camera-motion edits, it is preserved directly. Finally, the triplets undergo post-processing and human filtering based on alignment, consistency, rationality, and visual quality. The full data generation pipeline is shown in the figure below:

The system builds upon Helios-Distilled, a 14B autoregressive video diffusion transformer that generates video chunk by chunk. It maintains a hierarchical multi-scale memory to keep the computational cost bounded. The authors attach a lightweight Edit-Ignition Adapter to the frozen backbone to inject edit instructions without degrading the generation prior. The adapter is inserted after each transformer layer and operates on the per-layer hidden states $H = [H_{\mathrm{hist}}; H_{\mathrm{cur}}]$H=[Hhist​;Hcur​]. It refines the current-chunk tokens in three stages:

1. History Cross-Attention (HISTCA): Anchors the denoising chunk to the provided history by attending to history tokens.
2. Temporal Causal Self-Attention (TEMPSA): Propagates the anchored signal forward along the temporal axis using a causal mask.
3. Edit Cross-Attention (EDITCA): Injects the edit instruction $c_{\mathrm{edit}}$cedit​ into all denoising tokens. Each stage uses a modulation $m_{\sigma}(\cdot)$mσ​(⋅) based on the noise level $\sigma$σ, and output projections are zero-initialized to start as identity maps, ensuring the original generation behavior remains intact.

The adapter is trained using a flow-matching objective while keeping the base weights frozen. To mitigate exposure bias from imperfect generated history during inference, the authors apply history corruption during training, randomly replacing clean history latents with noisy versions. Furthermore, instead of the backbone's pyramid denoising schedule, they use a single-stage fixed-step Euler schedule with a mixture-Gaussian sampling of noise levels $\sigma$σ. This focuses the adapter on the specific $\sigma$σ values used during inference. Training follows a two-phase curriculum: the first phase uses uniform mixture weights and flat frame weights for broad coverage, while the second phase shifts weights toward lower $\sigma$σ values and later frames to refine fine details.

At inference, the adapter is used only to ignite the edit on the first chunk after an instruction arrives, while subsequent chunks are generated by the frozen backbone. The full inference pipeline is shown in the figure below:

To ensure fine details are captured, an extra denoising step at a near-zero $\sigma$σ is added for the ignition chunk. For infinite editing, the system employs a sliding history window and resets the anchor frame $x_0$x0​ to the first edited frame of the ignition chunk. This moving anchor provides a stable reference for the current edit, preventing drift and allowing the backbone to carry the edited content forward indefinitely.

Experiment

The authors evaluate a streaming editing method built on a frozen Helios-Distilled 14B backbone by training only an edit adapter and testing it on a newly constructed out-of-distribution sequential editing benchmark with 200 source videos and three-step edit chains. Comparisons against pure backbone, in-place editing, and prompt switching baselines show that the proposed approach performs best overall, especially in camera control, temporal stability, faithfulness, visual quality, scene preservation, and cross-edit coherence. It also remains stable across multiple editing rounds and sustains quality in long videos exceeding 1000 frames, where edited attributes persist across segments. The qualitative results further confirm that edits occur at the intended point while preserving source content and stream continuity.

The proposed method achieves the best camera motion and temporal flickering scores among the compared approaches, while remaining close to the top motion smoothness result. Its advantage is largest for camera motion, indicating stronger control over directional camera edits. Temporal flickering on the final editing round suggests maintained coherence after accumulated sequential edits. The proposed method leads in camera motion and temporal flickering, with a wide margin over the second-best camera motion score. Motion smoothness stays near the best result, while dynamic degree is reported only as a content-dependent reference rather than an edit-fidelity measure.

The proposed method ranks first on all four VLM-as-Judge dimensions for infinite video editing, with the largest advantages in scene identity preservation and visual quality. It substantially outperforms both in-place editing and prompt-switching baselines, and is the only approach that simultaneously delivers high edit faithfulness, content preservation, and cross-edit coherence across three editing rounds. The method achieves a wide lead in scene identity preservation and visual quality, with scores far above the next-best competitor. Only this method keeps each edit faithful, preserves unchanged source attributes, and maintains a continuous stream through all three edits, consistent with its top coherence and preservation scores.

The proposed method achieves the highest edit faithfulness across all three sequential editing rounds, staying near 3.8 with minimal variation (standard deviation 0.023). Other methods either start low and fluctuate strongly, or show misleading upward trends that actually reflect content drift rather than genuine editing quality. This stability confirms robust sequential editing without degradation. The proposed method maintains near-constant edit faithfulness (approximately 3.8) from the first to the third edit, with a standard deviation of only 0.023. Prompt switching methods exhibit extreme instability: faithfulness starts below 1.7 and jumps above 3.1, resulting in standard deviations over 0.69.

The experiments evaluate the proposed method on motion and temporal quality metrics, as well as on VLM-based judgments for infinite and sequential video editing. The method leads in camera motion control and temporal flickering while staying competitive in motion smoothness, and it ranks first on scene identity preservation and visual quality. Across repeated editing rounds, it maintains consistently high edit faithfulness without the instability or content drift seen in baselines, making it the only approach that reliably preserves unchanged attributes while applying faithful sequential edits.