Efficient 3D Semantic Segmentation with Superpoint Transformer

2023/06/13 by Robert, Damien, Raguet, Hugo, Landrieu, Loic · 11 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2306.08045

Abstract

We introduce a novel superpoint-based transformer architecture for efficient semantic segmentation of large-scale 3D scenes. Our method incorporates a fast algorithm to partition point clouds into a hierarchical superpoint structure, which makes our preprocessing 7 times faster than existing superpoint-based approaches. Additionally, we leverage a self-attention mechanism to capture the relationships between superpoints at multiple scales, leading to state-of-the-art performance on three challenging benchmark datasets: S3DIS (76.0% mIoU 6-fold validation), KITTI-360 (63.5% on Val), and DALES (79.6%). With only 212k parameters, our approach is up to 200 times more compact than other state-of-the-art models while maintaining similar performance. Furthermore, our model can be trained on a single GPU in 3 hours for a fold of the S3DIS dataset, which is 7x to 70x fewer GPU-hours than the best-performing methods. Our code and models are accessible at github.com/drprojects/superpoint_transformer.

Cited by

• LitePT: Lighter Yet Stronger Point Transformer
• EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation
• Foundry: Distilling 3D Foundation Models for the Edge
• CloudMamba: Grouped Selective State Spaces for Point Cloud Analysis
• TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding
• From Propagation to Prediction: Point-level Uncertainty Evaluation of MLS Point Clouds under Limited Ground Truth
• Point-level Uncertainty Evaluation of Mobile Laser Scanning Point Clouds
• COS3D: Collaborative Open-Vocabulary 3D Segmentation
• HARP-NeXt: High-Speed and Accurate Range-Point Fusion Network for 3D LiDAR Semantic Segmentation
• SGS-3D: High-Fidelity 3D Instance Segmentation via Reliable Semantic Mask Splitting and Growing
• Multispectral LiDAR data for extracting tree points in urban and suburban areas

Related

• Superpoint Transformer for 3D Scene Instance Segmentation
• Fast Point Transformer
• EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation
• Augmented Efficiency: Reducing Memory Footprint and Accelerating Inference for 3D Semantic Segmentation through Hybrid Vision
• GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds