One-sentence Summary

Researchers from Adobe and KAIST introduce MoviEdit, an end-to-end framework for long-form multi-shot video editing that combines automatic data-generation pipelines producing 87K paired editing videos with a cascaded bidirectional-autoregressive design and MoviEdit-Bench, substantially outperforming open-source models and matching industry frontier editors in text-guided and image-guided editing.

Key Contributions

• The paper introduces automatic data-generation pipelines that combine synchronized multi-view footage, real-world multi-shot videos, and generated multi-shot content to create 87K paired editing videos without manual annotation.
• MoviEdit uses a cascaded bidirectional-autoregressive design in which a lightweight bidirectional stage first establishes a globally coherent editing draft over a spatiotemporally compressed representation, then an autoregressive stage propagates and refines that draft over the full video.
• The paper contributes MoviEdit-Bench for systematic long-form multi-shot editing evaluation; experiments show MoviEdit substantially outperforms existing open-source models and performs on par with industry frontier models on text-guided and image-guided editing.

Introduction

Long-form video editing matters because visual narratives derive meaning from relationships across shots, so an edit must persist coherently through cuts even as scene, viewpoint, motion, and appearance change. Prior video editing methods are mostly designed for short single-shot clips; editing shots independently discards cross-shot dependencies, while jointly processing full source and target sequences becomes computationally prohibitive as duration grows. The authors identify three coupled bottlenecks in existing systems: scarce paired training data with explicit cross-shot subject correspondence, memory-heavy modeling under token concatenation and global self-attention, and the lack of dedicated benchmarks for cross-shot localization and consistency. To address these issues, they propose MoviEdit, an end-to-end framework with a scalable data generation pipeline combining multi-view-aware image editing, single-shot video editing, and VLM-based filtering; a cascaded bidirectional-autoregressive diffusion architecture that first builds a globally coherent edit draft and then propagates it causally; and MoviEdit-Bench, a dedicated benchmark with automated metrics for long-form multi-shot editing. This provides supervision, architecture, and evaluation infrastructure for moving video editing beyond isolated clips toward coherent long-form narratives.

Dataset

Training paired data

• The authors generate paired supervision without human annotation from three sources: in-house synchronized multi-camera footage, real-world cinematic clips, and a pretrained multi-shot text-to-video diffusion model.
• These sources support three supervision regimes: pixel-aligned editing, subject localization, and semi-pixel-aligned editing.
• Multi-camera footage provides cross-view correspondence, cinematic clips provide realism, and the diffusion model provides controllable synthesis.

Pixel-aligned editing pairs

• Clips are collected from cinematic sources through shot detection and scene grouping, or from synchronized multi-camera footage through camera-view selection and temporal cuts.
• A vision-language model proposes candidate subject queries and merges observations of the same entity across shots.
• Supported edits include object-level changes such as add, remove, swap, and appearance change, plus global changes such as environment change and stylization.
• The first frames from all shots are arranged into a single image grid and edited jointly, then each edited first frame is propagated within its shot.
• Quality filtering rejects 17% of multi-camera pairs at the first-frame stage and 38% at the propagation stage. For cinematic pairs, the rejection rates are 32% and 47%.

Subject localization pairs

• Video segmentation models track the queried subject across all shots where it appears and produce dense per-frame masks.
• Supported localized effects include highlighting, outlining, and color filling.
• Vision-language model filtering rejects 24% of generated pairs.

Semi-pixel-aligned editing pairs

• A pretrained text-driven multi-shot video generator is used for environment change, medium change, and character appearance change.
• Source and target generations share per-shot captions and initial noise, while an LLM creates distinct global captions that modify only the intended attribute.
• For character appearance change, source and target share a denoising trajectory until a fork timestep.
• For environment change and medium change, the target is conditioned on HED edge or depth maps extracted from the source through ControlNet.
• Vision-language model quality control rejects 66% of pairs. For ControlNet-based pairs, source and target are swapped when forming training pairs.

Usage

• The pixel-aligned editing task is the primary training setting.
• The generated multi-shot clips and subject queries are used to build paired source-target supervision for the editing model.
• Detailed training-data statistics are provided in the appendix, with data-recipe ablations reported separately.

MoviEdit-Bench

• The benchmark contains 64 generated multi-shot videos and 128 editing scenarios.
• Videos are organized into eight categories with varying cross-shot subject-binding difficulty, from single recurring subjects to crowded multi-subject and mixed human-animal scenes.
• It supports text-guided video editing and edited-frame-guided propagation, with first-frame and non-first-frame reference scenarios.
• Local edits include add, remove, appearance change, and swap. Global edits include environment change and stylization.
• Evaluation covers seven dimensions: non-subject preservation, edit localization, subject preservation, cross-shot edit consistency, edit-prompt alignment, video quality, and edited-frame fidelity for propagation.
• Source and edited videos are spatially and temporally aligned, and temporal metrics are computed independently within each shot.
• The benchmark combines pixel-level metrics, embedding-based similarity, and vision-language model judgments.

Method

The authors develop an end-to-end framework for long-form multi-shot video editing, beginning with scalable paired-data generation pipelines that construct supervision without human annotation. These pipelines leverage diverse video sources to support three distinct supervision regimes: pixel-aligned editing, subject localization, and semi-pixel-aligned editing.

For the pixel-aligned editing task, the authors address the critical challenge of cross-shot consistency by decomposing the process into two subproblems: cross-shot spatial editing and within-shot temporal editing. First frames from all source shots are arranged into a single image grid and jointly edited in one pass using an off-the-shelf text-guided image editing model $g_{\phi}$gϕ​. Each edited first frame is then propagated independently within its corresponding shot using a pretrained first-frame-conditioned single-shot video editing model $h_{\psi}$hψ​:

where $e$e denotes the edit instruction. Quality filtering is applied at both stages to ensure edit success and consistency. The subject localization pipeline utilizes video segmentation models to track queried subjects across shots and produce dense per-frame masks, applying localized visual transformations such as highlighting or outlining. The semi-pixel-aligned editing pipeline generates pairs that preserve spatial structure and narrative correspondence without strict pixel-level alignment. It couples source and target generation by sharing per-shot captions and initial noise. For character appearance changes, the source and target share a common denoising trajectory until a fork timestep $t_{\mathrm{fork}}$tfork​, where the shared latent is updated using averaged velocity predictions:

For environment and medium changes, the target generation is conditioned via ControlNet on structural maps extracted from the source video. The authors demonstrate that incorporating these specific data recipes significantly improves model generalization and edit localization capabilities.

To model long-form videos, the authors employ a cascaded architecture designed to mitigate the cumulative appearance drift and oversaturation inherent in standard chunk-wise autoregressive diffusion. Naive autoregressive rollouts often suffer from degraded visual quality over long horizons, a problem effectively resolved by introducing a global editing draft.

The architecture consists of a Draft Editor and a Render Editor that share model weights but differ in attention patterns and spatiotemporal resolutions. The Draft Editor applies a standard DiT-based denoiser $f_{\theta}$fθ​ bidirectionally to a spatially and temporally subsampled video sequence, producing a globally coherent edit draft. This draft serves as a stable appearance anchor and establishes cross-shot consistency. The Render Editor then performs full-resolution chunk-wise causal diffusion conditioned on this draft.

During training, the Render Editor is optimized using the causal diffusion forcing (CDF) objective:

where the causal self-attention mask $\mathcal{M}_{\mathrm{causal}}$Mcausal​ ensures each chunk is predicted using only itself and preceding chunks within the context window. During inference, the Render Editor generates chunks autoregressively, conditioning on the recently generated context, the global draft, and the corresponding source chunk.

To reduce inference latency, the authors apply unified distillation with asymmetric step schedules. Following velocity moment matching, the Draft Editor is distilled to 12 denoising steps, while the computationally heavier Render Editor is distilled to 4 steps. Additionally, the autoregressive formulation and causal attention masking naturally support KV caching, allowing the system to cache key-value states for reusable tokens and eliminate redundant computation during the rendering phase.

Experiment

The work introduces MoviEdit-Bench, a long-form multi-shot video editing benchmark that evaluates text-guided and edited-frame-guided editing across diverse subjects, local and global edits, and multiple consistency dimensions. Experiments show MoviEdit achieves the strongest overall text-guided performance and the highest overall edited-frame-guided success, with better source preservation, edit localization, and prompt alignment than public and proprietary baselines, which often propagate artifacts or deviate from source motion and timing. Scaling and efficiency studies indicate consistent gains from more data and training, while optimized attention and distillation substantially reduce inference latency, and ablations confirm that semi-pixel-aligned data helps structural edits and subject-localization supervision improves target identification. Additional applications demonstrate generalization to multi-reference editing, video set-to-set editing, and longer sequences, though ambiguous similar subjects and partial target visibility remain key failure cases.

MoviEdit achieves the strongest overall performance on long-form multi-shot text-guided video editing, surpassing Runway Aleph 2.0 in overall edit success and leading on most evaluation metrics. Among the listed baselines, Gemini Omni Flash shows the highest overall edit success and strong subject preservation, while Wan-Edit leads open-source models on non-subject preservation and edit localization. Public baselines commonly exhibit failures such as propagated artifacts, incorrect cross-shot edit localization, and poor source consistency. MoviEdit outperforms publicly available and proprietary systems on 15-second multi-shot V2V editing, surpassing Runway Aleph 2.0 in overall edit success. Wan-Edit achieves the best open-source non-subject preservation and edit-localization scores, while Gemini Omni Flash records the highest overall edit success among listed baselines.

For edited-frame-guided propagation, MoviEdit achieves the highest overall success on the full benchmark and performs comparably to Aleph 2.0 on individual metrics, with complementary strengths across dimensions. Aleph 2.0 is the strongest baseline overall, while HappyHorse shows consistently weaker non-subject preservation, edit localization, and cross-shot edit consistency. GenProp exhibits strong localization precision but trails in recall and fidelity metrics. MoviEdit leads in overall edit success for edited-frame-guided propagation and performs on par with Aleph 2.0 across individual evaluation dimensions. HappyHorse lags in non-subject preservation, edit localization, and edit consistency, whereas GenProp shows high precision but lower recall and edited-frame fidelity.

The training set combines multi-camera localization, multi-camera pixel-aligned, cinematic pixel-aligned, and generative semi-pixel-aligned editing pairs at 540p with 10 to 15 second videos and 4 to 6 shots per video. Filtering reduces roughly 205K raw pairs to about 87K pairs, with cinematic pixel-aligned data forming the largest share and generative semi-pixel-aligned data the smallest. Retention varies by pipeline, with subject localization pairs kept at the highest rate. Cinematic pixel-aligned pairs contribute more than half of the final filtered dataset. Subject localization pairs have the highest retention, keeping over three quarters of raw pairs. Generative semi-pixel-aligned pairs are the smallest group and are reduced by about two thirds during filtering. Overall, roughly two of every five raw pairs remain after filtering.

Inference latency ablations show that each optimization step reduces total denoising time, with the render editor contributing most of the cost. Sliding-window attention and KV caching provide progressive speedups, and adding few-step distillation produces the largest gain, bringing total time below the input video duration. The full temporal attention baseline is slowest, with render editor time dominating the total. Sliding-window attention reduces render editor latency, and KV caching further lowers it by a larger margin. Combining few-step distillation with attention and caching yields roughly a 31x total speedup and reduces total time to about 12 seconds.

MoviEdit is evaluated on long-form multi-shot text-guided video editing and edited-frame-guided propagation, where it achieves the strongest overall edit success and performs comparably to proprietary Runway Aleph 2.0 across individual metrics. Public baselines often suffer from propagated artifacts, incorrect cross-shot edit localization, and poor source consistency, while Wan-Edit leads open-source models on non-subject preservation and edit localization. The training pipeline filters roughly 205K raw multi-shot editing pairs to about 87K, with cinematic pixel-aligned data contributing the largest share and subject localization pairs retaining the highest rate. Inference ablations show that sliding-window attention, KV caching, and few-step distillation progressively reduce denoising time, yielding roughly a 31x total speedup to about 12 seconds.