Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders

Learning 3D representation plays a critical role in masked autoencoder (MAE)based pre-training methods for point cloud, including single-modal andcross-modal based MAE. Specifically, although cross-modal MAE methods learnstrong 3D representations via the auxiliary of other modal knowledge, theyoften suffer from heavy computational burdens and heavily rely on massivecross-modal data pairs that are often unavailable, which hinders theirapplications in practice. Instead, single-modal methods with solely pointclouds as input are preferred in real applications due to their simplicity andefficiency. However, such methods easily suffer from limited 3D representationswith global random mask input. To learn compact 3D representations, we proposea simple yet effective Point Feature Enhancement Masked Autoencoders(Point-FEMAE), which mainly consists of a global branch and a local branch tocapture latent semantic features. Specifically, to learn more compact features,a share-parameter Transformer encoder is introduced to extract point featuresfrom the global and local unmasked patches obtained by global random and localblock mask strategies, followed by a specific decoder to reconstruct.Meanwhile, to further enhance features in the local branch, we propose a LocalEnhancement Module with local patch convolution to perceive fine-grained localcontext at larger scales. Our method significantly improves the pre-trainingefficiency compared to cross-modal alternatives, and extensive downstreamexperiments underscore the state-of-the-art effectiveness, particularlyoutperforming our baseline (Point-MAE) by 5.16%, 5.00%, and 5.04% in threevariants of ScanObjectNN, respectively. The code is available athttps://github.com/zyh16143998882/AAAI24-PointFEMAE.