SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization

We propose a novel 3D point cloud segmentation framework named SASO, whichjointly performs semantic and instance segmentation tasks. For semanticsegmentation task, inspired by the inherent correlation among objects inspatial context, we propose a Multi-scale Semantic Association (MSA) module toexplore the constructive effects of the semantic context information. Forinstance segmentation task, different from previous works that utilizeclustering only in inference procedure, we propose a Salient Point ClusteringOptimization (SPCO) module to introduce a clustering procedure into thetraining process and impel the network focusing on points that are difficult tobe distinguished. In addition, because of the inherent structures of indoorscenes, the imbalance problem of the category distribution is rarely consideredbut severely limits the performance of 3D scene perception. To address thisissue, we introduce an adaptive Water Filling Sampling (WFS) algorithm tobalance the category distribution of training data. Extensive experimentsdemonstrate that our method outperforms the state-of-the-art methods onbenchmark datasets in both semantic segmentation and instance segmentationtasks.