DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF

2025/07/19 by Petit, Doriand, Bourgeois, Steve, Gay-Bellile, Vincent +2 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2507.14596

Abstract

3D semantic segmentation provides high-level scene understanding for applications in robotics, autonomous systems, etc. Traditional methods adapt exclusively to either task-specific goals (open-vocabulary segmentation) or scene content (unsupervised semantic segmentation). We propose DiSCO-3D, the first method addressing the broader problem of 3D Open-Vocabulary Sub-concepts Discovery, which aims to provide a 3D semantic segmentation that adapts to both the scene and user queries. We build DiSCO-3D on Neural Fields representations, combining unsupervised segmentation with weak open-vocabulary guidance. Our evaluations demonstrate that DiSCO-3D achieves effective performance in Open-Vocabulary Sub-concepts Discovery and exhibits state-of-the-art results in the edge cases of both open-vocabulary and unsupervised segmentation.

Citations

• EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentation
• GARField: Group Anything with Radiance Fields
• DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision
• CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentation
• SANeRF-HQ: Segment Anything for NeRF in High Quality
• Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding
• U3DS³: Unsupervised 3D Semantic Scene Segmentation
• Selective Visual Representations Improve Convergence and Generalization for Embodied AI
• SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation
• Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds
• Interactive Segment Anything NeRF with Feature Imitation
• Weakly Supervised 3D Open-vocabulary Segmentation
• DINOv2: Learning Robust Visual Features without Supervision
• Micrograph segmentations for DDEVD
• Segment Anything
• Generalized Decoding for Pixel, Image, and Language
• OpenScene: 3D Scene Understanding with Open Vocabularies
• ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation
• Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations
• Open-Vocabulary Universal Image Segmentation with MaskCLIP
• Decomposing NeRF for Editing via Feature Field Distillation
• Unsupervised Semantic Segmentation by Distilling Feature Correspondences
• Instant neural graphics primitives with a multiresolution hash encoding
• Language-driven Semantic Segmentation
• Scaling Open-Vocabulary Image Segmentation with Image-Level Labels
• Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
• Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
• Emerging Properties in Self-Supervised Vision Transformers
• In-Place Scene Labelling and Understanding with Implicit Scene Representation
• Learning Transferable Visual Models From Natural Language Supervision
• Deep Semantic Segmentation of Natural and Medical Images: A Review
• The Replica Dataset: A Digital Replica of Indoor Spaces
• Panoptic Segmentation

Cited by

• LLaVA³: Representing 3D Scenes like a Cubist Painter to Boost 3D Scene Understanding of VLMs

Related

• Open-NeRF: Towards Open Vocabulary NeRF Decomposition
• GOV-NeSF: Generalizable Open-Vocabulary Neural Semantic Fields
• Weakly Supervised 3D Open-vocabulary Segmentation
• Search3D: Hierarchical Open-Vocabulary 3D Segmentation
• NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes