One-sentence Summary

Researchers from Beijing Jiaotong University, Mootion AI, and other institutions propose StateFlow, a state-centric previsualization framework that maintains an explicit, persistent, editable 3D world state and employs a three-stage process of constructing, evolving, and accessing it through prior-guided, conflict-aware dual-view initialization, structured state transitions that preserve world memory, and render-feedback camera planning, thus enabling iterative, controllable scene editing for video creation and game prototyping.

Key Contributions

• The paper presents StateFlow, a state-centric generative previsualization framework that maintains a persistent, editable 3D world state of object geometry, poses, semantics, and cameras instead of generating videos in one shot.
• StateFlow contributes a three-stage pipeline: prior-guided conflict-aware dual-view initialization constructs coherent 3D scenes from generated 2D content; intent-guided state transitions evolve scenes while preserving world memory; and render-feedback reflection converts camera plans into visually feasible trajectories.
• Experiments demonstrate that StateFlow produces high-quality 3D worlds for video creation and game-like prototyping, with coherent spatial layout, stable scene structure, and editable object-level organization.

Introduction

Previsualization is an essential planning phase in filmmaking, game development, and architectural design, where creators iteratively explore scene layout, object placement, motion, and camera setups to communicate creative intent before final production. Existing generative approaches are largely one-shot: a single prompt must simultaneously specify all scene content, layout, motion, and cameras, with no persistent state to support local edits or consistent multi-view access, leading to spatiotemporal inconsistency and identity drift. While 3D scene assembly methods can build coherent static scenes, they treat construction as an endpoint and do not jointly model the world’s evolution or repeated camera-controlled access needed for previsualization. The authors introduce StateFlow, a state-centric framework that reformulates generative previsualization as building and interacting with an editable, evolvable 3D world state. The framework uses prior-guided dual-view initialization to construct the scene, intent-guided state transitions to evolve it, and render-feedback camera planning to access it, enabling downstream applications such as video creation and 3D game prototyping.

Method

The authors present StateFlow, a state-centric framework for previsualization that treats the process not as one-shot output generation, but as maintaining a persistent world state. They formulate previsualization as a state-based process where a system builds an initial world state from input conditions, updates it over time, and produces observations under controllable access:

$$\mathcal {W} ^ {0} = \mathcal {F} _ {\text {build}} (\mathcal {C}), \quad \mathcal {W} ^ {t + 1} = \mathcal {F} _ {\text {evolve}} (\mathcal {W} ^ {t}), \quad y ^ {t} = \mathcal {F} _ {\text {access}} (\mathcal {W} ^ {t})$$W0=Fbuild​(C),Wt+1=Fevolve​(Wt),yt=Faccess​(Wt)

Here, $\mathcal {W} ^ {t}$Wt denotes the world state at time $t$t, and $y ^ {t}$yt is the resulting observation. This formulation separates the world state, its evolution, and the rendered observation, which is crucial for creators who repeatedly modify the same world and inspect it from different viewpoints.

StateFlow instantiates $\mathcal {W} ^ {t}$Wt as an object-centric structured 3D state. At time $t$t, the world is represented as:

$$\mathcal {W} ^ {t} = \{o _ {i} ^ {t} \} _ {i = 1} ^ {N _ {t}}, \quad o _ {i} ^ {t} = (g _ {i} ^ {t}, p _ {i} ^ {t}, s _ {i} ^ {t})$$Wt={oit​}i=1Nt​​,oit​=(git​,pit​,sit​)

Here, $o _ {i} ^ {t}$oit​ denotes the $i$i-th object entity represented by its geometry $g _ {i} ^ {t}$git​, spatial placement $p _ {i} ^ {t}$pit​, and semantic attributes $s _ {i} ^ {t}$sit​. Built on this representation, the framework contains three stages: State Construction, State Evolution, and State Access.

As shown in the figure below:

The State Construction stage initializes the first world state $\mathcal {W} ^ {0}$W0 by estimating object geometry and global spatial layout. To combine complementary cues, the authors generate a front-view image for object assets and a bird's-eye view (BEV) image for spatial layout. Since these views are generated independently, they use a Vision-Language Model (VLM) to reconcile cross-view conflicts through a Prior-Guided Conflict-Aware Dual-View Initialization process. The front view serves as the appearance source, while the BEV acts as the spatial source. To lift BEV layouts into 3D placements, the VLM predicts a grounding prior $\gamma _ {i} \in \{\text{grounded, floating}\}$γi​∈{grounded, floating} for each object. Given a BEV box $\hat {r} _ {i} ^ {\mathrm{BEV}}$r^iBEV​ and a size prior $\bar {s} _ {i}$sˉi​, they initialize a 3D box using a grounding-aware lifting operator:

$$b _ {i} ^ {(0)} = \mathrm{Lift} _ {\gamma_ {i}} (\hat {r} _ {i} ^ {\mathrm{BEV}}, \bar {s} _ {i})$$bi(0)​=Liftγi​​(r^iBEV​,sˉi​)

Finally, they refine the retained boxes with a lightweight inference-time objective:

$$\mathcal {B} ^ {\star} = \arg \min _ {\mathcal {B}} \mathcal {L} _ {\text {front}} + \lambda_ {b} \mathcal {L} _ {\text {bev}} + \lambda_ {v} \mathcal {L} _ {\text {vlm}} + \lambda_ {p} \mathcal {L} _ {\text {phys}}$$B⋆=argBmin​Lfront​+λb​Lbev​+λv​Lvlm​+λp​Lphys​

This optimization updates only box parameters, keeping the construction lightweight and training-free. The resulting optimized 3D boxes, combined with geometry from an image-to-3D model and semantic attributes, form the initial world state $\mathcal {W} ^ {0}$W0.

The State Evolution stage updates the world state into $\mathcal {W} ^ {t + 1}$Wt+1 according to user intent via an Intent-Guided Structured State Transition. Instead of relying on a video renderer to hallucinate dynamics, a VLM queries the structured state table and predicts a compact transition plan $\Delta _ {t}$Δt​ specifying the objects or scene attributes to update. At the scene level, this supports global evolution like scene expansion and style changes. At the object level, it performs category-aware updates over the object state, handling character pose updates, rigid object motion, or event-level asset substitution for complex deformations. Formally, the next state is obtained by applying the plan to the structured state table:

$$\Delta_ {t} = \mathrm{Plan} _ {\mathrm{VLM}} (\mathcal {W} ^ {t}, u ^ {t}), \quad \mathcal {W} ^ {t + 1} = \mathrm{Apply} (\mathcal {W} ^ {t}, \Delta_ {t}) = \{o _ {i} ^ {t + 1} \} _ {i = 1} ^ {N _ {t + 1}}$$Δt​=PlanVLM​(Wt,ut),Wt+1=Apply(Wt,Δt​)={oit+1​}i=1Nt+1​​

The State Access stage determines how the constructed world is used for downstream tasks, primarily through camera trajectories. The authors introduce World-State Camera Planning with Render-Feedback Reflection to combine VLM semantic reasoning with geometric evidence. For each segment, the VLM acts as a semantic proposer to generate an initial trajectory from the world state, rendered observations $V$V, and director intent $d _ {i}$di​:

$$\pi_ {i} ^ {0} = \mathrm{Propose} _ {\mathrm{VLM}} (\mathcal {W} ^ {t}, V, d _ {i})$$πi0​=ProposeVLM​(Wt,V,di​)

They then execute the proposal in the 3D world through low-cost rendering and evaluate the result:

$$R _ {i} ^ {k} = \mathrm{Render} (\mathcal {W} ^ {t}, \pi_ {i} ^ {k}), \qquad e _ {i} ^ {k} = \mathrm{Eval} (R _ {i} ^ {k}, \pi_ {i} ^ {k}, d _ {i}, \mathcal {W} ^ {t})$$Rik​=Render(Wt,πik​),eik​=Eval(Rik​,πik​,di​,Wt)

Reflection converts discrepancies like intent mismatch or collision risks into local repair candidates $\Delta _ {m}$Δm​ over camera parameters. These repairs generate a compact candidate set, from which the best trajectory is selected:

$$\mathcal {P} _ {i} ^ {k} = \left\{\pi_ {i} ^ {k} + \Delta_ {m} \right\} _ {m = 1} ^ {M}, \quad \pi_ {i} ^ {k + 1} = \arg \min _ {\pi \in \mathcal {P} _ {i} ^ {k}} J \left(\pi; d _ {i}, \mathcal {W} ^ {t}, R _ {i} ^ {k}\right)$$Pik​={πik​+Δm​}m=1M​,πik+1​=argπ∈Pik​min​J(π;di​,Wt,Rik​)

This local proposal-and-verification loop iterates until no significant issue is detected, providing state access for keyframe generation and free-view exploration while remaining grounded in the persistent 3D world state.

Experiment

The system is evaluated through scene generation comparisons against 3D baselines, video creation comparisons against leading video generation models, and a user study with MLLM-based assessment. Ablation studies further validate the importance of bird's-eye-view layout grounding, conflict resolution, structured state transitions, and render-feedback camera planning. Results show that constructing a persistent 3D world state yields more coherent spatial layouts, better object consistency, and more stable camera trajectories, making StateFlow particularly suitable for previsualization tasks.

The proposed method attains the highest average VBench score, outperforming all compared video generation approaches. It leads in subject consistency, background consistency, motion smoothness, and flicker, indicating superior temporal stability and scene structure preservation. While some baselines show higher aesthetic and imaging quality, these metrics reflect low-level visual preferences, and the method excels on the structural and temporality dimensions most critical for controllable scene generation. the method achieves the best average score (0.8484) and top performance in subject consistency, background consistency, motion smoothness, and flicker. Animaker scores highest on aesthetic quality (0.6795), but its subject consistency and background consistency are notably lower than ours. Wan2.2 and Seedance2.0 show strong background consistency, yet still trail the method on that metric and on motion smoothness. Imaging quality is comparable between the method and the top baselines, with Seedance2.0 and MovieAgent only marginally ahead. The results highlight a trade-off: the method prioritizes structural and temporal coherence over low-level visual appeal, unlike baselines that favor aesthetic or imaging scores.

StateFlow attains the highest CLIP-I and CLIP-T scores, demonstrating strong visual and textual alignment with prompts. SynCity leads in HPS and Q-Align Aesthetics, though the paper notes these metrics may favor colorfulness and visual style. Other baselines, particularly SAM3D, show weaker text alignment despite relatively high quality scores. StateFlow achieves the best CLIP-I and CLIP-T, indicating superior image and text alignment compared to all baselines. SynCity records the highest HPS and Q-Align Aesthetics, which may be inflated by the metrics' preference for colorful and stylized outputs. SAM3D produces relatively high Q-Align scores but the lowest CLIP-T, revealing a gap between aesthetic quality and text alignment.

StateFlow outperforms baselines at the scene level across prompt alignment, layout plausibility, completeness, geometric quality, coherence, previsualization usefulness, and overall quality. In video-level evaluation, it also achieves the highest overall score with clear advantages in spatial consistency, identity consistency, camera quality, and previsualization usefulness. Ablations indicate that BEV layout grounding, conflict resolution, structured state updates, and render-feedback camera planning all contribute to coherent, controllable, and production-oriented results. StateFlow achieves the best scene-level performance across all evaluated dimensions, including layout plausibility, coherence, and overall quality. Video-level results show StateFlow leads in spatial consistency, identity consistency, camera quality, and previsualization usefulness, while some baselines remain competitive mainly on prompt or story alignment.

The experiments evaluate the proposed StateFlow method against video generation baselines using automated metrics and human assessments. StateFlow achieves the highest average VBench score and the best CLIP-I and CLIP-T results, demonstrating strong structural consistency, temporal smoothness, and visual text alignment, while some baselines score higher on aesthetic or imaging quality but show weaker consistency and alignment. Scene level and video level evaluations further show StateFlow leading in layout plausibility, coherence, spatial and identity consistency, camera quality, and previsualization usefulness. Ablations confirm that BEV layout grounding, conflict resolution, structured state updates, and render feedback camera planning each contribute to coherent and controllable generation.