MVGS: Multi-view-regulated Gaussian Splatting for Novel View Synthesis

Recent works in volume rendering, e.g. NeRF and 3D GaussianSplatting (3DGS), significantly advance the rendering quality and efficiencywith the help of the learned implicit neural radiance field or 3D Gaussians.Rendering on top of an explicit representation, the vanilla 3DGS and itsvariants deliver real-time efficiency by optimizing the parametric model withsingle-view supervision per iteration during training which is adopted fromNeRF. Consequently, certain views are overfitted, leading to unsatisfyingappearance in novel-view synthesis and imprecise 3D geometries. To solveaforementioned problems, we propose a new 3DGS optimization method embodyingfour key novel contributions: 1) We transform the conventional single-viewtraining paradigm into a multi-view training strategy. With our proposedmulti-view regulation, 3D Gaussian attributes are further optimized withoutoverfitting certain training views. As a general solution, we improve theoverall accuracy in a variety of scenarios and different Gaussian variants. 2)Inspired by the benefit introduced by additional views, we further propose across-intrinsic guidance scheme, leading to a coarse-to-fine training procedureconcerning different resolutions. 3) Built on top of our multi-view regulatedtraining, we further propose a cross-ray densification strategy, densifyingmore Gaussian kernels in the ray-intersect regions from a selection of views.4) By further investigating the densification strategy, we found that theeffect of densification should be enhanced when certain views are distinctdramatically. As a solution, we propose a novel multi-view augmenteddensification strategy, where 3D Gaussians are encouraged to get densified to asufficient number accordingly, resulting in improved reconstruction accuracy.