Dynamic Convolution for 3D Point Cloud Instance Segmentation

2021/07/18 by Tong He, Chunhua Shen, He, Tong +3 · 1 citation
Computer Science · Earth and Planetary Sciences · Engineering · Environmental Science · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Algorithm #Artificial intelligence #Artificial neural network #Bottleneck #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Convolution (computer science) #FOS: Computer and information sciences #Inference #Pattern recognition (psychology) #Point cloud #Remote Sensing and LiDAR Applications #Segmentation #Theoretical computer science #cs.CV

paper · pdf · doi:10.48550/arxiv.2107.08392

published in arXiv (Cornell University) (Cornell University) · Accepted to IEEE Trans. Pattern Analysis and Machine Intelligence. Extended version of arXiv:2011.13328

openalex publication_date 2021/07/18 · arxiv created 2022/10/14 · arxiv updated 2022/10/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose an approach to instance segmentation from 3D point clouds based on dynamic convolution. This enables it to adapt, at inference, to varying feature and object scales. Doing so avoids some pitfalls of bottom up approaches, including a dependence on hyper-parameter tuning and heuristic post-processing pipelines to compensate for the inevitable variability in object sizes, even within a single scene. The representation capability of the network is greatly improved by gathering homogeneous points that have identical semantic categories and close votes for the geometric centroids. Instances are then decoded via several simple convolution layers, where the parameters are generated conditioned on the input. The proposed approach is proposal-free, and instead exploits a convolution process that adapts to the spatial and semantic characteristics of each instance. A light-weight transformer, built on the bottleneck layer, allows the model to capture long-range dependencies, with limited computational overhead. The result is a simple, efficient, and robust approach that yields strong performance on various datasets: ScanNetV2, S3DIS, and PartNet. The consistent improvements on both voxel- and point-based architectures imply the effectiveness of the proposed method. Code is available at: https://git.io/DyCo3D

Citations

• Pay Less Attention with Lightweight and Dynamic Convolutions
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• A Simple Framework for Contrastive Learning of Visual Representations
• Improved Baselines with Momentum Contrastive Learning
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
• Momentum Contrast for Unsupervised Visual Representation Learning
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution

Related

• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution
• ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution
• PointInst3D: Segmenting 3D Instances by Points
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• CpT: Convolutional Point Transformer for 3D Point Cloud Processing