Jie-Ying Lee Yi-Ruei Liu Shr-Ruei Tsai Wei-Cheng Chang Chung-Ho Wu Jiewen Chan Zhenjun Zhao Chieh Hubert Lin Yu-Lun Liu

Abstract

Synthesizing large-scale, explorable, and geometrically accurate 3D urbanscenes is a challenging yet valuable task in providing immersive and embodiedapplications. The challenges lie in the lack of large-scale and high-qualityreal-world 3D scans for training generalizable generative models. In thispaper, we take an alternative route to create large-scale 3D scenes bysynergizing the readily available satellite imagery that supplies realisticcoarse geometry and the open-domain diffusion model for creating high-qualityclose-up appearances. We propose Skyfall-GS, the first city-blockscale 3D scene creation framework without costly 3D annotations, also featuringreal-time, immersive 3D exploration. We tailor a curriculum-driven iterativerefinement strategy to progressively enhance geometric completeness andphotorealistic textures. Extensive experiments demonstrate that Skyfall-GSprovides improved cross-view consistent geometry and more realistic texturescompared to state-of-the-art approaches. Project page:https://skyfall-gs.jayinnn.dev/