Self-Supervised Deep Learning on Point Clouds by Reconstructing Space

2019/01/24 by Jonathan Sauder, Sauder, Jonathan, Bjarne Sievers +1 · 1 citation
Computer Science · Engineering · Environmental Science · #3D Shape Modeling and Analysis #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Remote Sensing and LiDAR Applications

paper · pdf · doi:10.48550/arxiv.1901.08396

openalex publication_date 2019/01/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Point clouds provide a flexible and natural representation usable in countless applications such as robotics or self-driving cars. Recently, deep neural networks operating on raw point cloud data have shown promising results on supervised learning tasks such as object classification and semantic segmentation. While massive point cloud datasets can be captured using modern scanning technology, manually labelling such large 3D point clouds for supervised learning tasks is a cumbersome process. This necessitates methods that can learn from unlabelled data to significantly reduce the number of annotated samples needed in supervised learning. We propose a self-supervised learning task for deep learning on raw point cloud data in which a neural network is trained to reconstruct point clouds whose parts have been randomly rearranged. While solving this task, representations that capture semantic properties of the point cloud are learned. Our method is agnostic of network architecture and outperforms current unsupervised learning approaches in downstream object classification tasks. We show experimentally, that pre-training with our method before supervised training improves the performance of state-of-the-art models and significantly improves sample efficiency.

Citations

• The Earth Mover's Distance as a Metric for Image Retrieval
• Distributed Representations of Words and Phrases and their Compositionality
• Order Matters: Sequence to sequence for sets
• ShapeNet: An Information-Rich 3D Model Repository
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• A scalable active framework for region annotation in 3D shape collections
• Geometric Deep Learning: Going beyond Euclidean data
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Dynamic Graph CNN for Learning on Point Clouds

Cited by

• A Cross Branch Fusion-Based Contrastive Learning Framework for Point Cloud Self-supervised Learning

Related

• Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey
• Self-Supervised Few-Shot Learning on Point Clouds
• A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds
• Self-Supervised Learning for Point Clouds Data: A Survey
• Self-supervised Point Cloud Representation Learning via Separating Mixed Shapes