LOSC: LiDAR Open-voc Segmentation Consolidator

2025/07/10 by Samet, Nermin, Puy, Gilles, Marlet, Renaud
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2507.07605

Abstract

We study the use of image-based Vision-Language Models (VLMs) for open-vocabulary segmentation of lidar scans in driving settings. Classically, image semantics can be back-projected onto 3D point clouds. Yet, resulting point labels are noisy and sparse. We consolidate these labels to enforce both spatio-temporal consistency and robustness to image-level augmentations. We then train a 3D network based on these refined labels. This simple method, called LOSC, outperforms the SOTA of zero-shot open-vocabulary semantic and panoptic segmentation on both nuScenes and SemanticKITTI, with significant margins.

Citations

• Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems
• Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation
• Zero-Shot 4D Lidar Panoptic Segmentation
• LeAP: Consistent multi-domain 3D labeling using Foundation Models
• AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction
• Image Segmentation in Foundation Model Era: A Survey
• Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation
• 3D Annotation-Free Learning by Distilling 2D Open-Vocabulary Segmentation Models for Autonomous Driving
• When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models
• Better Call SAL: Towards Learning to Segment Anything in Lidar
• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
• Point Transformer V3: Simpler, Faster, Stronger
• Three Pillars improving Vision Foundation Model Distillation for Lidar
• CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free
• Towards Open Vocabulary Learning: A Survey
• Towards Label-free Scene Understanding by Vision Foundation Models
• Segment Everything Everywhere All at Once
• Micrograph segmentations for DDEVD
• Segment Anything
• Vision-Language Models for Vision Tasks: A Survey
• Vision-Language Models for Vision Tasks: A Survey
• A Simple Framework for Open-Vocabulary Segmentation and Detection
• Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation
• CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
• OpenScene: 3D Scene Understanding with Open Vocabularies
• LAION-5B: An open large-scale dataset for training next generation image-text models
• Open-Vocabulary Universal Image Segmentation with MaskCLIP
• Language-driven Semantic Segmentation
• Scaling Open-Vocabulary Image Segmentation with Image-Level Labels
• RegionCLIP: Region-based Language-Image Pretraining
• Extract Free Dense Labels from CLIP
• Learning Transferable Visual Models From Natural Language Supervision
• nuScenes: A multimodal dataset for autonomous driving
• Albumentations: Fast and Flexible Image Augmentations

Related

• KPRNet: Improving projection-based LiDAR semantic segmentation
• VolMap: A Real-time Model for Semantic Segmentation of a LiDAR surrounding view
• NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentation
• LEST: Large-scale LiDAR Semantic Segmentation with Transformer
• TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation