NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolation

In recent years, there has been a significant increase in focus on theinterpolation task of computer vision. Despite the tremendous advancement ofvideo interpolation, point cloud interpolation remains insufficiently explored.Meanwhile, the existence of numerous nonlinear large motions in real-worldscenarios makes the point cloud interpolation task more challenging. In lightof these issues, we present NeuralPCI: an end-to-end 4D spatio-temporal Neuralfield for 3D Point Cloud Interpolation, which implicitly integrates multi-frameinformation to handle nonlinear large motions for both indoor and outdoorscenarios. Furthermore, we construct a new multi-frame point cloudinterpolation dataset called NL-Drive for large nonlinear motions in autonomousdriving scenes to better demonstrate the superiority of our method. Ultimately,NeuralPCI achieves state-of-the-art performance on both DHB (Dynamic HumanBodies) and NL-Drive datasets. Beyond the interpolation task, our method can benaturally extended to point cloud extrapolation, morphing, and auto-labeling,which indicates its substantial potential in other domains. Codes are availableat https://github.com/ispc-lab/NeuralPCI.