CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence

Motivated by the intuition that the critical step of localizing a 2D image inthe corresponding 3D point cloud is establishing 2D-3D correspondence betweenthem, we propose the first feature-based dense correspondence framework foraddressing the image-to-point cloud registration problem, dubbed CorrI2P, whichconsists of three modules, i.e., feature embedding, symmetric overlappingregion detection, and pose estimation through the established correspondence.Specifically, given a pair of a 2D image and a 3D point cloud, we firsttransform them into high-dimensional feature space and feed the resultingfeatures into a symmetric overlapping region detector to determine the regionwhere the image and point cloud overlap each other. Then we use the features ofthe overlapping regions to establish the 2D-3D correspondence before runningEPnP within RANSAC to estimate the camera's pose. Experimental results on KITTIand NuScenes datasets show that our CorrI2P outperforms state-of-the-artimage-to-point cloud registration methods significantly. We will make the codepublicly available.