Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud

Recent advancements in self-supervised learning in the point cloud domainhave demonstrated significant potential. However, these methods often sufferfrom drawbacks, including lengthy pre-training time, the necessity ofreconstruction in the input space, or the necessity of additional modalities.In order to address these issues, we introduce Point-JEPA, a joint embeddingpredictive architecture designed specifically for point cloud data. To thisend, we introduce a sequencer that orders point cloud patch embeddings toefficiently compute and utilize their proximity based on the indices duringtarget and context selection. The sequencer also allows shared computations ofthe patch embeddings' proximity between context and target selection, furtherimproving the efficiency. Experimentally, our method achieves competitiveresults with state-of-the-art methods while avoiding the reconstruction in theinput space or additional modality.