LOSC: LiDAR Open-voc Segmentation Consolidator

2025/07/10 by Nermin Samet, Samet, Nermin, Gilles Puy +3 · 2 voices · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.2507.07605

arxiv published 2025/07/10 · arxiv updated 2026/03/13

Abstract

We study the use of image-based Vision-Language Models (VLMs) for open-vocabulary segmentation of lidar scans in driving settings. Classically, image semantics can be back-projected onto 3D point clouds. Yet, resulting point labels are noisy and sparse. We consolidate these labels to enforce both spatio-temporal consistency and robustness to image-level augmentations. We then train a 3D network based on these refined labels. This simple method, called LOSC, outperforms the SOTA of zero-shot open-vocabulary semantic and panoptic segmentation on both nuScenes and SemanticKITTI, with significant margins. Code is available at https://github.com/valeoai/LOSC.

Citations

• Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems
• Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation
• Zero-Shot 4D Lidar Panoptic Segmentation
• LeAP: Consistent multi-domain 3D labeling using Foundation Models
• AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction
• Image Segmentation in Foundation Model Era: A Survey
• Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation
• 3D Annotation-Free Learning by Distilling 2D Open-Vocabulary Segmentation Models for Autonomous Driving
• When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models
• Better Call SAL: Towards Learning to Segment Anything in Lidar
• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
• Point Transformer V3: Simpler, Faster, Stronger
• Three Pillars improving Vision Foundation Model Distillation for Lidar
• CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free
• Towards Open Vocabulary Learning: A Survey
• Towards Label-free Scene Understanding by Vision Foundation Models
• Segment Everything Everywhere All at Once
• Micrograph segmentations for DDEVD
• Segment Anything
• Vision-Language Models for Vision Tasks: A Survey
• Vision-Language Models for Vision Tasks: A Survey
• A Simple Framework for Open-Vocabulary Segmentation and Detection
• Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation
• CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
• OpenScene: 3D Scene Understanding with Open Vocabularies
• LAION-5B: An open large-scale dataset for training next generation image-text models
• Open-Vocabulary Universal Image Segmentation with MaskCLIP
• Language-driven Semantic Segmentation
• Scaling Open-Vocabulary Image Segmentation with Image-Level Labels
• RegionCLIP: Region-based Language-Image Pretraining
• Extract Free Dense Labels from CLIP
• Learning Transferable Visual Models From Natural Language Supervision
• nuScenes: A multimodal dataset for autonomous driving
• Albumentations: fast and flexible image augmentations

Cited by

• Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenization
• OccAny: Generalized Unconstrained Urban 3D Occupancy

Discussions

• 📄 Read the full paper: arxiv.org/abs/2507.07605 💻 Explore the code: github.com/valeoai/LOSC [bsky, 1 points, 0 comments]
• LOSC: LiDAR Open-voc Segmentation Consolidator 🚨oral🚨 tl;dr: 1) projecting 2D VLM predictions into 3D space + 2) refining w/ spatio-temp. and augmentation consistency --> 3) train a robust 3D networ [bsky, 0 points, 0 comments]

Related

• KPRNet: Improving projection-based LiDAR semantic segmentation
• VolMap: A Real-time Model for Semantic Segmentation of a LiDAR surrounding view
• NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentation
• LEST: Large-scale LiDAR Semantic Segmentation with Transformer
• TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation