Calibration of Model Uncertainty for Dropout Variational Inference

2020/06/20 by Max-Heinrich Laves, Laves, Max-Heinrich, Sontje Ihler +5 · 5 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2006.11584

openalex publication_date 2020/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. In this paper, different logit scaling methods are extended to dropout variational inference to recalibrate model uncertainty. Expected uncertainty calibration error (UCE) is presented as a metric to measure miscalibration. The effectiveness of recalibration is evaluated on CIFAR-10/100 and SVHN for recent CNN architectures. Experimental results show that logit scaling considerably reduce miscalibration by means of UCE. Well-calibrated uncertainty enables reliable rejection of uncertain predictions and robust detection of out-of-distribution data.

Citations

• Variational Dropout and the Local Reparameterization Trick
• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Measuring Calibration in Deep Learning
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Federated Learning via Meta-Variational Dropout
• Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction
• Weight Pruning and Uncertainty in Radio Galaxy Classification
• Uncertainty-aware Test-Time Training (UT³) for Efficient On-the-fly Domain Adaptive Dense Regression
• Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking

Related

• Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inference
• Improving model calibration with accuracy versus uncertainty optimization
• Unreliable Uncertainty Estimates with Monte Carlo Dropout
• Robustly representing uncertainty in deep neural networks through sampling
• Confidence Calibration for Convolutional Neural Networks Using Structured Dropout