Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering

2024/01/12 by Damien Robert, Robert, Damien, Hugo Raguet +3 · 3 citations
Computer Science · Engineering · Environmental Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications #Robotics and Sensor-Based Localization #Video Surveillance and Tracking Methods

paper · pdf · doi:10.48550/arxiv.2401.06704

openalex publication_date 2024/01/12 · openalex created_date 2024/01/16 · openalex updated_date 2026/07/28

Abstract

We introduce a highly efficient method for panoptic segmentation of large 3D point clouds by redefining this task as a scalable graph clustering problem. This approach can be trained using only local auxiliary tasks, thereby eliminating the resource-intensive instance-matching step during training. Moreover, our formulation can easily be adapted to the superpoint paradigm, further increasing its efficiency. This allows our model to process scenes with millions of points and thousands of objects in a single inference. Our method, called SuperCluster, achieves a new state-of-the-art panoptic segmentation performance for two indoor scanning datasets: 50.1 PQ (+7.8) for S3DIS Area~5, and 58.7 PQ (+25.2) for ScanNetV2. We also set the first state-of-the-art for two large-scale mobile mapping benchmarks: KITTI-360 and DALES. With only 209k parameters, our model is over 30 times smaller than the best-competing method and trains up to 15 times faster. Our code and pretrained models are available at https://github.com/drprojects/superpoint_transformer.

Citations

• Optimal approximations by piecewise smooth functions and associated variational problems
• Some generalized order-disorder transformations
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Deep Learning for 3D Point Clouds: A Survey
• PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
• Point Transformer V2: Grouped Vector Attention and Partition-based Pooling

Cited by

• LitePT: Lighter Yet Stronger Point Transformer
• EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation
• Point Cloud Segmentation of Integrated Circuits Package Substrates Surface Defects Using Causal Inference: Dataset Construction and Methodology

Related

• Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs
• A Review of Panoptic Segmentation for Mobile Mapping Point Clouds
• Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization
• Point Cloud Oversegmentation with Graph-Structured Deep Metric Learning
• EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation