Learning Depth Estimation for Transparent and Mirror Surfaces

2023/07/27 by Alex Costanzino, Costanzino, Alex, Pierluigi Zama Ramirez +9 · 13 citations
Computer Science · Earth and Planetary Sciences · #3D Surveying and Cultural Heritage #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Optical measurement and interference techniques

paper · pdf · doi:10.48550/arxiv.2307.15052

openalex publication_date 2023/07/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Inferring the depth of transparent or mirror (ToM) surfaces represents a hard challenge for either sensors, algorithms, or deep networks. We propose a simple pipeline for learning to estimate depth properly for such surfaces with neural networks, without requiring any ground-truth annotation. We unveil how to obtain reliable pseudo labels by in-painting ToM objects in images and processing them with a monocular depth estimation model. These labels can be used to fine-tune existing monocular or stereo networks, to let them learn how to deal with ToM surfaces. Experimental results on the Booster dataset show the dramatic improvements enabled by our remarkably simple proposal.

Cited by

• Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation
• Rectifying Latent Space for Generative Single-Image Reflection Removal
• DepthFocus: Controllable Depth Estimation for See-Through Scenes
• EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects
• HDCNet: A Hybrid Depth Completion Network for Grasping Transparent and Reflective Objects
• Real-Time Glass Detection and Reprojection using Sensor Fusion Onboard Aerial Robots
• BRIDGE -- Building Reinforcement-Learning Depth-to-Image Data Generation Engine for Monocular Depth Estimation
• S\textsuperscript2M\textsuperscript2: Scalable Stereo Matching Model for Reliable Depth Estimation
• NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfaces
• HTMNet: A Hybrid Network with Transformer-Mamba Bottleneck Multimodal Fusion for Transparent and Reflective Objects Depth Completion
• GIFT: Geometry-Invariant Fine-Tuning for Non-Lambertian Monocular Depth Estimation
• Multi-Label Stereo Matching for Transparent Scene Depth Estimation
• The Fourth Monocular Depth Estimation Challenge

Related

• Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces
• Towards Robust Monocular Depth Estimation in Non-Lambertian Surfaces
• Seeing and Seeing Through the Glass: Real and Synthetic Data for Multi-Layer Depth Estimation
• Self-Supervised Learning for Transparent Object Depth Completion Using Depth from Non-Transparent Objects
• Learning Monocular Depth Estimation via Selective Distillation of Stereo Knowledge