Enhancing 3D Semantic Scene Completion with a Refinement Module

2025/12/20 by Dunxing Zhang, Zhang, Dunxing, Jiachen Lu +7
Engineering · Computer Science · #3D Shape Modeling and Analysis #Advanced Neural Network Applications #Generative Adversarial Networks and Image Synthesis

paper · doi:10.48550/arxiv.2512.18363

Abstract

We propose ESSC-RM, a plug-and-play Enhancing framework for Semantic Scene Completion with a Refinement Module, which can be seamlessly integrated into existing SSC models. ESSC-RM operates in two phases: a baseline SSC network first produces a coarse voxel prediction, which is subsequently refined by a 3D U-Net-based Prediction Noise-Aware Module (PNAM) and Voxel-level Local Geometry Module (VLGM) under multiscale supervision. Experiments on SemanticKITTI show that ESSC-RM consistently improves semantic prediction performance. When integrated into CGFormer and MonoScene, the mean IoU increases from 16.87% to 17.27% and from 11.08% to 11.51%, respectively. These results demonstrate that ESSC-RM serves as a general refinement framework applicable to a wide range of SSC models.

Citations

• Vision-based 3D Semantic Scene Completion via Capture Dynamic Representations
• VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion
• VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion
• jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images
• Multimodal Remote Sensing Scene Classification Using VLMs and Dual-Cross Attention Networks
• Multimodal Alignment and Fusion: A Survey
• TALoS: Enhancing Semantic Scene Completion via Test-time Adaptation on the Line of Sight
• Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection
• Jina CLIP: Your CLIP Model Is Also Your Text Retriever
• Context and Geometry Aware Voxel Transformer for Semantic Scene Completion
• Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation
• Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion
• Long-CLIP: Unlocking the Long-Text Capability of CLIP
• SemCity: Semantic Scene Generation with Triplane Diffusion
• Faster Neighborhood Attention: Reducing the O(n²) Cost of Self Attention at the Threadblock Level
• EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
• Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
• PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness
• DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation
• Improved Baselines with Visual Instruction Tuning
• Symphonize 3D Semantic Scene Completion with Contextual Instance Queries
• SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving
• Contrastive Grouping with Transformer for Referring Image Segmentation
• InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
• Visual Instruction Tuning
• OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction
• Micrograph segmentations for DDEVD
• EVA-CLIP: Improved Training Techniques for CLIP at Scale
• GPT-4 Technical Report
• SCPNet: Semantic Scene Completion on Point Cloud
• Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen
• VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
• Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Dilated Neighborhood Attention Transformer
• 3D Object Detection for Autonomous Driving: A Comprehensive Survey
• Neighborhood Attention Transformer
• 3D Object Detection from Images for Autonomous Driving: A Survey
• Language-driven Semantic Segmentation
• BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
• MonoScene: Monocular 3D Semantic Scene Completion
• DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries
• Semantic Segmentation-assisted Scene Completion for LiDAR Point Clouds
• 3D Semantic Scene Completion: a Survey
• Learning Transferable Visual Models From Natural Language Supervision
• Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
• Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion
• LMSCNet: Lightweight Multiscale 3D Semantic Completion
• AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics
• Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by\n Implicitly Unprojecting to 3D
• Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
• DSGN: Deep Stereo Geometry Network for 3D Object Detection
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
• Deep Learning for 3D Point Clouds: A Survey
• Deep Learning for 3D Point Clouds: A Survey
• M3D-RPN: Monocular 3D Region Proposal Network for Object Detection
• Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
• CenterNet: Keypoint Triplets for Object Detection
• SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences
• Stereo R-CNN based 3D Object Detection for Autonomous Driving
• Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
• PointPillars: Fast Encoders for Object Detection from Point Clouds
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
• ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape
• Joint Monocular 3D Vehicle Detection and Tracking
• Pyramid Stereo Matching Network
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• Decoupled Weight Decay Regularization
• Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
• Attention Is All You Need
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• PointNet: Deep Learning on Point Sets for 3D Classification and\n Segmentation
• 3D Bounding Box Estimation Using Deep Learning and Geometry
• Semantic Scene Completion from a Single Depth Image
• Instance Normalization: The Missing Ingredient for Fast Stylization
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Empirical Evaluation of Rectified Activations in Convolutional Network

Related

• Semantic Scene Completion with Cleaner Self
• CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion
• Unleashing Network Potentials for Semantic Scene Completion
• Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective
• 3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior