Unsupervised Point Cloud Pre-Training via Occlusion Completion

2020/10/02 by Hanchen Wang, Qi Liu, Wang, Hanchen +7 · 25 citations
Computer Science · Earth and Planetary Sciences · Engineering · Mathematics · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Artificial intelligence #Code (set theory) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Construct (python library) #Downstream (manufacturing) #Encoder #FOS: Computer and information sciences #Image Processing and 3D Reconstruction #Machine Learning (cs.LG) #Mathematics #Maxima and minima #Object (grammar) #Pattern recognition (psychology) #Point (geometry) #Point cloud #Range (aeronautics) #Segmentation #Set (abstract data type) #Training (meteorology) #Transformation (genetics) #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2010.01089

published in arXiv (Cornell University) (Cornell University) · sync with ICCV camera ready

openalex publication_date 2020/10/02 · arxiv created 2021/10/13 · arxiv updated 2021/10/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We describe a simple pre-training approach for point clouds. It works in three steps: 1. Mask all points occluded in a camera view; 2. Learn an encoder-decoder model to reconstruct the occluded points; 3. Use the encoder weights as initialisation for downstream point cloud tasks. We find that even when we construct a single pre-training dataset (from ModelNet40), this pre-training method improves accuracy across different datasets and encoders, on a wide range of downstream tasks. Specifically, we show that our method outperforms previous pre-training methods in object classification, and both part-based and semantic segmentation tasks. We study the pre-trained features and find that they lead to wide downstream minima, have high transformation invariance, and have activations that are highly correlated with part labels. Code and data are available at: https://github.com/hansen7/OcCo

Citations

• Strategies for Pre-training Graph Neural Networks
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• Momentum Contrast for Unsupervised Visual Representation Learning
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
• Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
• Self-Supervised Pretraining of 3D Features on any Point-Cloud

Cited by

• Dual-Branch Center-Surrounding Contrast: Rethinking Contrastive Learning for 3D Point Clouds
• CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Images
• EBind: a practical approach to space binding
• GFT: Graph Feature Tuning for Efficient Point Cloud Analysis
• DAP-MAE: Domain-Adaptive Point Cloud Masked Autoencoder for Effective Cross-Domain Learning
• Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimation
• Unlocking Hidden Potential in Point Cloud Networks with Attention-Guided Grouping-Feature Coordination
• Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Driving
• Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views
• Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis
• Masked Clustering Prediction for Unsupervised Point Cloud Pre-training
• Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion
• UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting
• Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding
• Gaussian2Scene: 3D Scene Representation Learning via Self-supervised Learning with 3D Gaussian Splatting
• Attention-Guided Multi-Scale Local Reconstruction for Point Clouds via Masked Autoencoder Self-Supervised Learning
• LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds
• Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning
• GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
• A Cross Branch Fusion-Based Contrastive Learning Framework for Point Cloud Self-supervised Learning
• PMA: Towards Parameter-Efficient Point Cloud Understanding via Point Mamba Adapter
• HyMamba: Mamba with Hybrid Geometry-Feature Coupling for Efficient Point Cloud Classification
• PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning
• KAN or MLP? Point Cloud Shows the Way Forward
• 3D CoCa: Contrastive Learners are 3D Captioners

Related

• Occlusion-aware Text-Image-Point Cloud Pretraining for Open-World 3D Object Recognition
• PRED: Pre-training via Semantic Rendering on LiDAR Point Clouds
• PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generation
• Self-Supervised Pre-training of 3D Point Cloud Networks with Image Data
• Self-supervised Point Cloud Representation Learning via Separating Mixed Shapes