Improving Open-Set Semantic Segmentation in 3D Point Clouds by Conditional Channel Capacity Maximization: Preliminary Results

2025/05/09 by Wang Fang, Fang, Wang, Rahimi, Shirin +11
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Robotics and Sensor-Based Localization #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.11521

openalex publication_date 2025/05/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Point-cloud semantic segmentation underpins a wide range of critical applications. Although recent deep architectures and large-scale datasets have driven impressive closed-set performance, these models struggle to recognize or properly segment objects outside their training classes. This gap has sparked interest in Open-Set Semantic Segmentation (O3S), where models must both correctly label known categories and detect novel, unseen classes. In this paper, we propose a plug and play framework for O3S. By modeling the segmentation pipeline as a conditional Markov chain, we derive a novel regularizer term dubbed Conditional Channel Capacity Maximization (3CM), that maximizes the mutual information between features and predictions conditioned on each class. When incorporated into standard loss functions, 3CM encourages the encoder to retain richer, label-dependent features, thereby enhancing the network's ability to distinguish and segment previously unseen categories. Experimental results demonstrate effectiveness of proposed method on detecting unseen objects. We further outline future directions for dynamic open-world adaptation and efficient information-theoretic estimation.

Citations

• Towards Undistillable Models by Minimizing Conditional Mutual Information
• Distributed Quasi-Newton Method for Fair and Fast Federated Learning
• Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers
• PDF: A Probability-Driven Framework for Open World 3D Point Cloud Semantic Segmentation
• Robustness Against Adversarial Attacks via Learning Confined Adversarial Polytopes
• Conditional Mutual Information Constrained Deep Learning for Classification
• Novel Class Discovery for 3D Point Cloud Semantic Segmentation
• PointCaM: Cut-and-Mix for Open-Set Point Cloud Learning
• APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation
• 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks
• KPConv: Flexible and Deformable Convolution for Point Clouds
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• A scalable active framework for region annotation in 3D shape collections

Related

• PDF: A Probability-Driven Framework for Open World 3D Point Cloud Semantic Segmentation
• Open-world Point Cloud Semantic Segmentation: A Human-in-the-loop Framework
• Few to Big: Prototype Expansion Network via Diffusion Learner for Point Cloud Few-shot Semantic Segmentation
• GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds
• Deep Metric Learning for Open World Semantic Segmentation