One-sentence Summary

Researchers from Westlake University, Zhejiang University, and University College London propose Three-Body Scattering Modeling (TBSM), a one-step generative framework that uses per-projectile attraction-repulsion interactions to induce motion aligned with the $2\text{-Wasserstein gradient flow}$2-Wasserstein gradient flow of $\frac{1}{2} D_E^2(P_\theta, Q)$21​DE2​(Pθ​,Q), and by tracking this conditional expectation online to reduce field noise, TBSM trains generators on ImageNet-256 to achieve FID 2.23 with PixelDiT-XL and 1.63 with DiT-XL at a single sampling step.

Key Contributions

• TBSM converts the energy distance into constant-size sample-level supervision without adversarial critics, prescribed noise-to-data paths, or autoregressive factorizations. Each projectile is attracted toward a real sample and repelled from a generated one, enabling PixelDiT-XL to achieve FID 2.23 and DiT-XL to achieve FID 1.63 on ImageNet-256 at NFE=1.
• Online tracked scattering replaces noisy instantaneous field estimates with a learned conditional expectation tracker, reducing sampling variance. Under explicit assumptions, it provides error decompositions, finite-error stationarity, and energy-distance convergence, with matched-update configurations favoring tracked over instant scattering.
• An interpretive design map connects TBSM to Drift-like dynamics, GAN-like displacement analogies, and diffusion-related supervision via input and target changes, situating sample-level scattering as a framework linking explicit distributional motion, representation-space matching, and critic-based objectives.

Introduction

Generative models that aim for one-step sampling typically depend on adversarial training, diffusion distillation with teacher networks, or minibatch distribution matching, all of which bring challenges such as discriminator instability, expensive multi-step teacher queries, or large batch requirements. The authors introduce Three-Body Scattering Modeling (TBSM), which converts the energy distance into a constant-size per-sample interaction: each generated sample is attracted to one real observation and repelled from an independently generated sample, yielding model–model repulsion without any teacher, adversary, or denoising target. This sample-level supervision naturally handles text–image datasets with a single image per caption, and when combined with online tracking of the resulting noisy vector field it achieves competitive FID scores on high-resolution ImageNet generation at a single network evaluation.

Method

The authors formulate the learning goal as condition-wise distribution matching by minimizing the expected squared energy distance between the generated distribution $P_{\boldsymbol{\theta}}(\cdot \mid \mathbf{c})$Pθ​(⋅∣c) and the real distribution $Q(\cdot \mid \mathbf{c})$Q(⋅∣c). The first variation of this functional yields a population vector field, which dictates the generated-particle scattering vector field. This field is defined as the difference between the expected inter-source bearing towards real samples and the expected intra-source bearing towards generated samples.

To estimate this vector field efficiently, the authors introduce a three-body scattering estimator. For a given condition, the method samples a projectile from the generator, a real source from the data distribution, and an independently generated source. The sample-level scattering estimator is computed as the difference between the inter-source bearing and the intra-source bearing.

For the generator update, the authors employ scattering target regression. They displace the projectile by the sampled scattering vector to construct a detached target. The generator is then trained to regress its output toward this fixed target. This frozen-target regression serves as a local loss surrogate, where its expected gradient at the current parameters exactly matches the gradient of the energy-distance objective.

Because the single-sample estimator can be noisy, the authors introduce a tracker network to approximate the conditional expectation of the scattering field online. The tracker is queried along a corridor between the generated and real samples. The final target vector is a mixture of the instantaneous stochastic vector and the learned tracker field, controlled by a tracked-supervision weight $\rho$ρ. This mixing allows the method to trade off instantaneous variance against tracker error.

The method is organized within a generative design map parameterized by $(\rho, \lambda)$(ρ,λ), where $\lambda$λ jointly sets the intra-source coefficient in the scattering estimator and the tracker-query range.

This map highlights four reference configurations. At $\rho=0, \lambda=1$ρ=0,λ=1, the method uses instant scattering, resembling drift-like particle dynamics. At $\rho=1, \lambda=1$ρ=1,λ=1, it uses tracked scattering at the energy-distance endpoint, which yields the best sample quality by replacing source-sampling noise with a learned field. At $\rho=0, \lambda=0$ρ=0,λ=0, the update reduces to inter-only scattering, which resembles diffusion-related objectives but lacks the full denoising structure. Finally, at $\rho=1, \lambda=0$ρ=1,λ=0, the method follows a fake-to-real scattering path, connecting to GAN-like interpolation where the tracker learns from unit-bearing targets queried along the fake-to-real corridor.

For practical deployment on natural images, the authors apply representation-space scattering. They pass the projectile, real source, and generated source through a frozen encoder and compute the scattering loss in the feature space. Gradients backpropagate through the encoder into the generator, while source features remain detached. The tracker is parameterized as a lightweight vector-valued network to avoid the computational cost of higher-order autograd required by conservative scalar potential networks.

Experiment

The evaluation focuses on one-step ImageNet-256 generation with TBSM, initializing from pretrained multistep models and using representation fields from frozen encoders. Multi-encoder fields consistently improve quality, and the method achieves competitive FID scores across pixel and latent backbones, with λ tuning producing a guidance-like trade-off. Qualitative observations reveal that metric improvements do not always align with visual artifact reduction, and the approach is limited by the encoder's retained information and untested random initialization scalability.

The table compares major generative paradigms on three axes: whether they support direct single-pass sampling, the strength of their empirical scaling evidence, and the granularity of generator supervision. GANs are the only established paradigm that combines direct generation with per-sample feedback, but their scaling evidence is mixed; diffusion and autoregressive models exhibit strong scaling evidence at the cost of iterative sampling. Direct distribution dynamics methods achieve one-step generation with growing scaling evidence, though they typically rely on batch-estimated supervision rather than per-sample targets. GANs uniquely provide direct single-pass generation and per-sample supervision from a learned critic, but their scaling evidence is mixed. Diffusion and autoregressive models have strong scaling evidence but require iterative sampling, preventing single-pass generation. One-step diffusion acceleration methods show variable sampling efficiency and growing scaling evidence, with supervision granularity that depends on the specific technique. Direct distribution dynamics methods achieve direct generation and growing scaling evidence, yet their supervision is batch-estimated rather than sample-level.

Combining multiple frozen feature spaces consistently improves image quality over any single encoder. Any ensemble that includes SigLIP2-B outperforms all individual encoders across FID, FDr⁶, and IS. The SigLIP2-B + MAE-B pair achieves the best perceptual scores at a moderate per-step cost, while adding ResNet-18 yields the lowest FID at the expense of higher computation. All encoder combinations containing SigLIP2-B surpass every single encoder on FID, FDr⁶, and IS. The SigLIP2-B + MAE-B pair gives the best FDr⁶ (21.80) and IS (161.36) among all configurations, with a lower per-step runtime than the three-encoder field. The three-encoder field (ResNet-18 + MAE-B + SigLIP2-B) achieves the lowest FID (8.29) but has the highest average step time. Among single encoders, MAE-B yields the lowest FDr⁶ (27.74), SigLIP2-B yields the highest IS (131.49) but worst FID, and ResNet-18 sits in between. Adding a third encoder to SigLIP2-B + MAE-B trades higher per-step cost for a further reduction in FID, while IS slightly decreases.

One-step TBSM generators approach the quality of multi-step diffusion models on ImageNet, with latent-space DiT-XL/2 outperforming pixel-space PixelDiT-XL. Lowering the intra-source weight λ at a fixed scattering radius ρ mimics classifier-free guidance by simultaneously improving FID and Inception Score on both JiT-B and PixelDiT-XL backbones. TBSM also supports compact pixel architectures and, with a high-resolution curriculum, yields strong one-step results at 512×512 resolution. A single network evaluation (NFE=1) with TBSM reaches competitive FID: 1.63 in latent space (DiT-XL/2) and 2.23 in pixel space (PixelDiT-XL). At fixed ρ=0.9, reducing λ from 1.0 to 0.9 lowers FID (e.g., 3.35 to 2.92 on JiT-B) and raises Inception Score, an empirical tuning effect analogous to classifier-free guidance. The small JiT-B pixel backbone achieves one-step FID of 2.92 with (ρ=0.9, λ=0.9), demonstrating compatibility with lightweight generators. A high-resolution curriculum allows one-step TBSM to reach FID 1.92 with latent DiT-XL/4 and FID 3.84 with PixelDiT-XL on ImageNet-512. TBSM samples exhibit fewer block-like and grid-like artifacts than some multi-step baselines despite worse quantitative metrics, indicating a possible metric–artifact mismatch.

Three experiments are presented: a paradigm comparison table, a study on combining frozen feature encoders, and an evaluation of one-step TBSM generators. The paradigm analysis reveals that GANs uniquely offer direct single-pass generation with per-sample supervision but lack strong scaling evidence, while diffusion and autoregressive models scale well at the cost of iterative sampling. The encoder study shows that ensembles including SigLIP2-B consistently improve image quality over single encoders, with the SigLIP2-B + MAE-B pair optimizing perceptual scores and a three-encoder field achieving the lowest FID at higher computational cost. The TBSM experiments demonstrate that one-step generators can approach multi-step diffusion quality, with latent-space models outperforming pixel-space ones, and that tuning a weight parameter mimics classifier-free guidance while reducing artifacts despite worse quantitative metrics.