SG-PGM: Partial Graph Matching Network with Semantic Geometric Fusion for 3D Scene Graph Alignment and Its Downstream Tasks

Scene graphs have been recently introduced into 3D spatial understanding as acomprehensive representation of the scene. The alignment between 3D scenegraphs is the first step of many downstream tasks such as scene graph aidedpoint cloud registration, mosaicking, overlap checking, and robot navigation.In this work, we treat 3D scene graph alignment as a partial graph-matchingproblem and propose to solve it with a graph neural network. We reuse thegeometric features learned by a point cloud registration method and associatethe clustered point-level geometric features with the node-level semanticfeature via our designed feature fusion module. Partial matching is enabled byusing a learnable method to select the top-k similar node pairs. Subsequentdownstream tasks such as point cloud registration are achieved by running apre-trained registration network within the matched regions. We further proposea point-matching rescoring method, that uses the node-wise alignment of the 3Dscene graph to reweight the matching candidates from a pre-trained point cloudregistration method. It reduces the false point correspondences estimatedespecially in low-overlapping cases. Experiments show that our method improvesthe alignment accuracy by 10~20% in low-overlap and random transformationscenarios and outperforms the existing work in multiple downstream tasks.