Reconstructing 4D Spatial Intelligence: A Survey

Reconstructing 4D spatial intelligence from visual observations has long beena central yet challenging task in computer vision, with broad real-worldapplications. These range from entertainment domains like movies, where thefocus is often on reconstructing fundamental visual elements, to embodied AI,which emphasizes interaction modeling and physical realism. Fueled by rapidadvances in 3D representations and deep learning architectures, the field hasevolved quickly, outpacing the scope of previous surveys. Additionally,existing surveys rarely offer a comprehensive analysis of the hierarchicalstructure of 4D scene reconstruction. To address this gap, we present a newperspective that organizes existing methods into five progressive levels of 4Dspatial intelligence: (1) Level 1 -- reconstruction of low-level 3D attributes(e.g., depth, pose, and point maps); (2) Level 2 -- reconstruction of 3D scenecomponents (e.g., objects, humans, structures); (3) Level 3 -- reconstructionof 4D dynamic scenes; (4) Level 4 -- modeling of interactions among scenecomponents; and (5) Level 5 -- incorporation of physical laws and constraints.We conclude the survey by discussing the key challenges at each level andhighlighting promising directions for advancing toward even richer levels of 4Dspatial intelligence. To track ongoing developments, we maintain an up-to-dateproject page: https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence.