One-sentence Summary

Researchers from the University of British Columbia, MIT CSAIL, Georgia Institute of Technology, et al. propose WorldParticle, a unified transformer-based Lagrangian particle simulator that uses a prediction-correction design with particle tokenization, hierarchical super-token encoding and decoding, and progressive token merging to model cloth, elastic solids, Newtonian and non-Newtonian fluids, granular materials, and molecular dynamics while reducing attention cost and enabling downstream control, inverse design, and learning from real-world manipulation data.

Key Contributions

• A unified transformer-based particle simulator models cloth, elastic solids, Newtonian and non-Newtonian fluids, granular materials, and molecular dynamics within a single architecture, reducing the need for per-phenomenon solver engineering.
• The method uses a prediction-correction framework on a shared Lagrangian particle representation, combining an explicit external-force predictor with a learned corrector that predicts residual position and velocity updates through particle tokenization, hierarchical super-token encoding with progressive token merging, and super-token decoding.
• The same architecture generalizes to unseen materials, boundary configurations, initial conditions, and external forces, and supports downstream tasks including interactive control, inverse design, and learning from real-world manipulation data.

Introduction

Simulating diverse physical phenomena in one framework is a long-standing goal in graphics, engineering, and molecular modeling, but traditional solvers remain specialized: fluids often use Eulerian grids, solids use Lagrangian finite-element meshes, and granular or atomistic systems use particles, which makes cross-phenomenon coupling difficult. Particle-based Lagrangian representations offer a common state format, yet they still depend on problem-specific constitutive laws and parameter tuning, and existing learned simulators are usually designed for one phenomenon or tied to a fixed geometry. The authors propose a particle-based prediction-correction transformer whose architecture stays fixed across six dynamics categories, including cloth, elastic solids, Newtonian and non-Newtonian fluids, granular materials, and molecular dynamics. A particle tokenizer encodes local interactions, while a multigrid-inspired super-token encoder-decoder compresses tokens and restores full particle resolution to provide global communication in every correction step.

Method

The authors model one step of Lagrangian particle dynamics as a prediction-correction procedure. An explicit prediction step integrates known external forces to produce an intermediate state that accounts for external acceleration but ignores inter-particle interactions. Let $X_t, V_t, F_t \in \mathbb{R}^{N \times 3}$Xt​,Vt​,Ft​∈RN×3 denote the positions, velocities, and external forces of the simulated particles, and let $C \in \mathbb{R}^{N \times C_p}$C∈RN×Cp​ denote their per-particle attributes. Static boundary particles $(X^B, C^B)$(XB,CB) represent walls or obstacles, and $T$T denotes rest-shape topology when available. The prediction step integrates the external forces over one timestep:

where $M$M is the diagonal mass matrix. A learned correction step then predicts the interaction correction by taking the predicted state and outputting residual updates:

yielding $X_{t+\Delta t} = \tilde{X}_t + \Delta X_t$Xt+Δt​=X~t​+ΔXt​ and $V_{t+\Delta t} = \tilde{V}_t + \Delta V_t$Vt+Δt​=V~t​+ΔVt​. Predicting both residuals allows the model to represent effects from implicit solves, contact projections, pressure coupling, and damping.

The corrector $\mathcal{F}_\theta$Fθ​ consists of three main components that separate communication into local and global scales. As shown in the figure below:

The particle tokenizer encodes local interactions, the super-token encoder compresses particle tokens into dynamically generated super tokens, and the super-token decoder lifts super-token information back to particle resolution to produce position and velocity corrections.

The particle tokenizer follows the locality principle underlying particle methods. For each particle, it evaluates interactions over compact neighborhoods: a spatial neighborhood from radius search, a boundary neighborhood from nearby boundary samples, and a topological neighborhood defined by rest-shape tessellation adjacency. Each branch aggregates neighbor contributions through a learnable kernel $W_k$Wk​ that maps a relative displacement to a mixing matrix. The particle token $h_i$hi​ concatenates all available branches with the particle's own state.

While particle tokens capture local interactions, many physical phenomena require global, long-range communication. The super-token encoder addresses this by alternating global self-attention at the current resolution with token merging, inspired by hierarchical coarsening principles. Each encoder layer first applies multi-head self-attention with 3D rotary position embeddings to contextualize all tokens. It then performs token merging, which halves the sequence length. The current tokens are split into two disjoint halves, and each token in the first half is greedily matched to its most cosine-similar token in the second half. Visualizing the token merging process:

The matched pairs are merged via weighted averaging, while unmatched tokens are kept as-is. Applying self-attention before merging ensures that similarity is computed after each token has seen current-scale context. After $L_e$Le​ layers, the remaining tokens form the super tokens, which act as a sparse, queryable latent representation of the current particle system.

The super-token decoder lifts this coarse context back to particle resolution through $L_d$Ld​ layers. The original particle tokens and positions from the tokenizer are passed directly to the decoder. At each decoder layer, particle tokens query the super tokens through cross-attention, followed by self-attention to refine the tokens. All attention blocks use 3D rotary position embeddings, residual connections, and feed-forward networks. A final particle-wise MLP maps the refined tokens to the position and velocity corrections.

For training, the authors use the same architecture across all dynamics categories and train one network per category. Starting from an initial ground-truth state, they unroll the prediction-correction update autoregressively for a rollout window of states, feeding each predicted state back as input. The predicted states are supervised against the ground truth states. The training loss combines the mean per-particle, per-step position and velocity error averaged over the rollout with an optional physical constraint loss, such as a divergence-free regularizer for incompressible fluids.

Experiment

The experiments evaluate the proposed particle dynamics model across six categories, including fluids, cloth, sand, non-Newtonian materials, elastic solids, and proteins, using data from established solvers and testing varied material parameters, initial conditions, and boundaries. Comparisons show that the model remains stable over long rollouts while baselines exhibit floating particles, cloth explosions, and interpenetration, and generalization experiments indicate robustness to unseen initial and boundary conditions, longer horizons, novel geometries, and different particle densities. Ablations confirm that combining local tokenization with global super-token communication is essential for accurate dynamics, and physics-informed regularization helps reduce divergence while maintaining trajectory quality. Additional applications demonstrate interactive force control, inverse design via differentiable rollouts, and learning from real-world point-cloud observations.

The experiments compare accuracy and inference cost across six particle dynamics categories. Cloth and protein simulations are relatively efficient and accurate, while fluid rollouts show substantially larger position and velocity errors than all other categories. Sand and non-Newtonian fluids incur the highest per-step compute cost and lowest throughput. Fluid dynamics has the largest rollout error by far, with position and velocity errors much higher than for the other materials. Cloth achieves the fastest inference and the lowest per-step compute cost among all evaluated categories. Protein dynamics are the most accurate in both position and velocity error among the tested systems. Sand and non-Newtonian fluids have the highest per-step compute cost and the lowest frame rates.

The experiments assess accuracy and inference cost across six particle dynamics categories. Cloth and protein are the most efficient and accurate, with cloth achieving the fastest inference and lowest per-step compute, while protein yields the best position and velocity accuracy. Fluid rollouts show by far the largest errors, and sand and non-Newtonian fluids incur the highest per-step compute cost and lowest frame rates.