DepthMatch: Semi-Supervised RGB-D Scene Parsing through Depth-Guided Regularization

2025/05/26 by Jianxin Huang, Huang, Jianxin, Jiahang Li +7
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Medical Image Segmentation Techniques

paper · pdf · doi:10.48550/arxiv.2505.20041

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

RGB-D scene parsing methods effectively capture both semantic and geometric features of the environment, demonstrating great potential under challenging conditions such as extreme weather and low lighting. However, existing RGB-D scene parsing methods predominantly rely on supervised training strategies, which require a large amount of manually annotated pixel-level labels that are both time-consuming and costly. To overcome these limitations, we introduce DepthMatch, a semi-supervised learning framework that is specifically designed for RGB-D scene parsing. To make full use of unlabeled data, we propose complementary patch mix-up augmentation to explore the latent relationships between texture and spatial features in RGB-D image pairs. We also design a lightweight spatial prior injector to replace traditional complex fusion modules, improving the efficiency of heterogeneous feature fusion. Furthermore, we introduce depth-guided boundary loss to enhance the model's boundary prediction capabilities. Experimental results demonstrate that DepthMatch exhibits high applicability in both indoor and outdoor scenes, achieving state-of-the-art results on the NYUv2 dataset and ranking first on the KITTI Semantics benchmark.

Citations

• UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation
• UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation
• RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion
• GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer
• Playing to Vision Foundation Model's Strengths in Stereo Matching
• Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
• SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection
• Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning
• DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation
• Effective Whole-body Pose Estimation with Two-stages Distillation
• DINOv2: Learning Robust Visual Features without Supervision
• Complementary Random Masking for RGB-Thermal Semantic Segmentation
• Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation
• CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers
• Warp-Refine Propagation: Semi-Supervised Auto-labeling via\n Cycle-consistency
• Vision Transformers for Dense Prediction
• Active Boundary Loss for Semantic Segmentation
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
• CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
• Improving Semantic Segmentation via Video Propagation and Label Relaxation
• Improved Regularization of Convolutional Neural Networks with Cutout
• Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Fully Convolutional Networks for Semantic Segmentation

Related

• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• Fully Exploiting Vision Foundation Model's Profound Prior Knowledge for Generalizable RGB-Depth Driving Scene Parsing
• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
• Consistent Depth Prediction under Various Illuminations using Dilated Cross Attention
• DFormerv2: Geometry Self-Attention for RGBD Semantic Segmentation