One-sentence Summary

University of Tübingen, ShanghaiTech University, and Czech Technical University in Prague propose Mip-Splatting, which introduces a 3D smoothing filter that constrains Gaussian primitive sizes based on the maximal sampling frequency induced by input views and replaces 2D dilation with a 2D Mip filter simulating a 2D box filter, thereby eliminating aliasing and dilation artifacts in novel view synthesis across scales.

Key Contributions

• Mip-Splatting adds a 3D smoothing filter to 3D Gaussian Splatting that constrains Gaussian primitive sizes using the maximal sampling frequency induced by the input views, reducing high-frequency artifacts when zooming.
• A 2D Mip filter replaces 2D dilation and approximates the physical box-filter imaging process, mitigating aliasing and dilation issues while allowing filter sizes to be determined from input pixel size rather than rendered-view size.
• In training on single-scale images and testing at multiple scales, the method outperforms state-of-the-art methods in out-of-distribution sampling-rate settings, improving generalization to unseen camera poses and zoom factors.

Introduction

Novel view synthesis is important for applications such as virtual reality, robotics, and cinematography because it enables new camera viewpoints to be generated from captured scene data. Recent methods like Neural Radiance Fields and 3D Gaussian Splatting achieve high-quality reconstruction and real-time rendering, but 3D Gaussian Splatting can produce visible artifacts when zooming in or out, especially when the rendering scale differs from the training views. The authors identify this issue as stemming from missing 3D frequency constraints and from the use of a fixed 2D dilation filter. To address it, they introduce a 3D smoothing filter that limits Gaussian primitive frequencies according to the Nyquist limit of the training images, along with a 2D Mip filter that better models the physical imaging process and reduces aliasing. Together, these changes improve 3DGS generalization across unseen scales and sampling rates.

Method

The proposed framework extends 3D Gaussian Splatting with sampling-aware filtering. In the baseline formulation, a scene is represented by a set of scaled 3D Gaussian primitives $\{\mathcal{G}_k\}_{k=1}^{K}${Gk​}k=1K​. Each primitive is parameterized by a center $\mathbf{p}_k$pk​, a covariance matrix $\boldsymbol{\Sigma}_k$Σk​, an opacity $\alpha_k$αk​, and a view-dependent color $\mathbf{c}_k$ck​. The covariance is constrained through the positive semi-definite parameterization

where $\mathbf{s}_k \in \mathbb{R}^{3}$sk​∈R3 controls scale and $\mathbf{O}_k \in \mathbb{R}^{3 \times 3}$Ok​∈R3×3 is a rotation matrix.

For a camera defined by rotation $\mathbf{R}$R and translation $\mathbf{t}$t, each Gaussian is first transformed into camera space:

The projected covariance is then approximated in screen space through a local affine transformation:

By discarding the third dimension of this matrix, the method obtains a 2D covariance $\boldsymbol{\Sigma}_k^{2D}$Σk2D​ and a corresponding 2D Gaussian $\mathcal{G}_k^{2D}$Gk2D​. Final rendering is performed by depth-ordered alpha blending:

The original method also applies a fixed screen-space dilation operator to avoid degenerate subpixel Gaussians:

This dilation increases the effective support of small projected Gaussians while keeping their peak response unchanged. The authors show that this design introduces a scale ambiguity when the camera sampling rate changes. During inverse rendering, several different 3D scales can produce nearly identical training-time appearances because the dilation operator masks small covariances. As illustrated in the figure below, faithful and degenerate 3D representations may render similarly at the original sampling rate, but zooming in or zooming out exposes high-frequency erosion artifacts or excessive brightness and dilation.

To address this issue, the method introduces two complementary components: a 3D smoothing filter and a 2D Mip filter. The 3D smoothing filter is motivated by the observation that a radiance field reconstructed from multi-view images cannot contain spatial frequencies higher than those sampled by the training images. For a camera with focal length $f$f in pixel units, a screen-space sampling interval of one pixel maps to a world-space interval at depth $d$d:

Thus, the recoverable frequency is bounded by $\hat{\nu}/2$ν^/2. Since a primitive may be observed by multiple cameras with different depths and focal lengths, the authors define a per-primitive maximal sampling rate:

where $\mathbb{1}_n(\mathbf{p}_k)$1n​(pk​) indicates whether the primitive center lies in the frustum of camera $n$n. This procedure effectively band-limits each Gaussian according to the camera that provides the finest sampling for that primitive. The idea is visualized in the following figure.

The regularized 3D primitive is obtained by convolving the original Gaussian with a low-pass Gaussian filter:

Because the convolution of two Gaussians yields another Gaussian with summed covariance, this operation can be implemented analytically:

This modification constrains the highest representable frequency of each Gaussian relative to the maximal sampling rate observed among training views. The filter is associated with the reconstructed geometry and can be fused into the 3D primitive after optimization.

The second modification replaces the baseline screen-space dilation with a 2D Mip filter that approximates the physical integration of light over a pixel area. Instead of merely inflating the projected Gaussian, the method uses a normalized 2D Gaussian with an expanded covariance:

Here, the scalar $s$s is chosen so that the filter covers approximately one pixel in screen space. This approximates the box-filter behavior of real imaging while retaining the efficiency of Gaussian evaluation. The authors distinguish this operator from a bandwidth-limiting EWA filter because the 2D Mip filter is intended to model pixel-area integration rather than empirically control frequency bandwidth.

During training, the full pipeline follows the standard 3DGS optimization workflow. Gaussian positions, covariances, opacities, and spherical-harmonic colors are optimized by backpropagating a multi-view photometric loss, and adaptive densification and pruning are used to adjust the number of primitives. The only additional operation is the periodic computation of each primitive’s sampling rate, which is recomputed every $m$m iterations because Gaussian centers evolve during optimization. Once the 3D smoothing filter is fused with the primitive covariances, rendering follows the same splatting workflow as the original method, with the 2D Mip filter applied during projection to produce sampling-rate-aware alpha compositing.

Experiment

The experiments implement the method on 3DGS with periodic sampling-rate updates and 2D and 3D filters, then evaluate it on Blender and Mip-NeRF 360 under multi-scale and single-scale settings. The Blender results validate strong multi-scale rendering and robustness when training at full resolution and testing at lower resolutions, where the method avoids dilation and oversmoothing seen in 3DGS-based baselines. The Mip-NeRF 360 tests further show that it generalizes better than prior methods at higher resolutions and matches 3DGS at the training scale, while a Gaussian-filter approximation and periodic sampling-rate updates remain minor limitations.

In a Blender evaluation with multi-scale training and multi-scale testing, the approach achieves comparable or superior performance to strong state-of-the-art methods across image quality metrics. It substantially outperforms 3DGS and 3DGS plus EWA, supporting the effectiveness of its 2D Mip filter. It matches or exceeds leading multi-resolution models in most metrics. It gives large quality gains over 3DGS-based methods across multiple resolution scales.

When models trained only on full-resolution Blender images are evaluated at lower resolutions, most baselines remain reasonable near the training scale but degrade as zoom-out increases. The proposed method is competitive at full resolution and better across reduced resolutions. 3DGS-based methods can capture detail at the training resolution but lose quality more sharply when tested at smaller scales. Lower test resolutions simulate zoom-out and generally make rendering quality harder to preserve. 3DGS performs well at full resolution but drops substantially as resolution decreases. EWA splatting can oversmooth images, especially at lower resolutions.

In single-scale training and multi-scale testing on Mip-NeRF 360, the proposed method matches prior methods at the training scale and gains more as rendered resolution increases. It maintains higher quality across zoomed renderings, where other methods degrade more sharply. The trend indicates better behavior at higher sampling rates without pronounced high-frequency artifacts. Comparable performance at the training scale Clear advantage at higher rendering scales Lower degradation across zoomed evaluations

In the standard Mip-NeRF 360 setting where models are trained and tested at the same scale, the proposed approach remains competitive with strong Gaussian splatting baselines and other neural rendering methods. Its quality metrics are on par with well-established techniques, indicating stable performance without degradation in this challenging in-distribution benchmark. The method performs on par with 3DGS and 3DGS with EWA. It maintains competitive image quality across PSNR, SSIM, and LPIPS. The result supports effectiveness in standard same-scale rendering settings.

The experiments on Blender and Mip-NeRF 360 evaluate multi-scale training and testing, single-scale training with multi-scale rendering, and standard same-scale rendering. The proposed 2D Mip filter provides comparable or better image quality than 3DGS-based baselines, especially when rendering away from the training resolution, where it reduces aliasing and degradation more effectively. In same-scale benchmarks, it remains competitive with strong neural rendering and Gaussian splatting methods, showing stable in-distribution performance.