Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

2018/11/30 by Jishnu Mukhoti, Yarin Gal, Mukhoti, Jishnu +1 · 10 citations
Computer Science · #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning and Data Classification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1811.12709

openalex publication_date 2018/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes. This information is critical when using semantic segmentation for autonomous driving for example. Standard semantic segmentation systems have well-established evaluation metrics. However, with BDL's rising popularity in computer vision we require new metrics to evaluate whether a BDL method produces better uncertainty estimates than another method. In this work we propose three such metrics to evaluate BDL models designed specifically for the task of semantic segmentation. We modify DeepLab-v3+, one of the state-of-the-art deep neural networks, and create its Bayesian counterpart using MC dropout and Concrete dropout as inference techniques. We then compare and test these two inference techniques on the well-known Cityscapes dataset using our suggested metrics. Our results provide new benchmarks for researchers to compare and evaluate their improved uncertainty quantification in pursuit of safer semantic segmentation.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Variational Dropout and the Local Reparameterization Trick
• Distributed Representations of Sentences and Documents
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• ImageNet classification with deep convolutional neural networks
• End to End Learning for Self-Driving Cars
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry\n and Semantics

Cited by

• Out-of-Distribution Segmentation via Wasserstein-Based Evidential Uncertainty
• Enhancing decision support in crop production: Analyzing conformal prediction for uncertainty quantification
• Uncertainty-Aware Deep Learning for Wildfire Danger Forecasting
• Probabilistic Semantic Segmentation Refinement by Monte Carlo Region Growing
• Approaching Neural Network Uncertainty Realism
• Uncertainty-Aware Physically-Guided Proxy Tasks for Unseen Domain Face Anti-spoofing
• Bayesian Graph Neural Networks with Adaptive Connection Sampling
• Road Obstacle Video Segmentation
• Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation
• TRADI: Tracking deep neural network weight distributions for uncertainty\n estimation

Related

• Evaluating Uncertainty Estimation Methods on 3D Semantic Segmentation of Point Clouds
• The Robust Semantic Segmentation UNCV2023 Challenge Results
• The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
• In Depth Bayesian Semantic Scene Completion