Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference

2018/06/15 by Kumar Shridhar, Shridhar, Kumar, Felix Laumann +3 · 3 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1806.05978

openalex publication_date 2018/06/15 · openalex created_date 2019/05/29 · openalex updated_date 2026/07/28

Abstract

We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate aleatoric and epistemic uncertainty in a coherent manner. The intractable posterior probability distributions over weights are inferred by Bayes by Backprop. Firstly, we demonstrate how this reliable variational inference method can serve as a fundamental construct for various network architectures. On multiple datasets in supervised learning settings (MNIST, CIFAR-10, CIFAR-100), this variational inference method achieves performances equivalent to frequentist inference in identical architectures, while the two desiderata, a measure for uncertainty and regularization are incorporated naturally. Secondly, we examine how our proposed measure for aleatoric and epistemic uncertainties is derived and validate it on the aforementioned datasets.

Citations

• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Variational Dropout and the Local Reparameterization Trick
• Improving neural networks by preventing co-adaptation of feature detectors
• On Information and Sufficiency
• Gradient-based learning applied to document recognition
• Auxiliary Deep Generative Models
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Bayesian Recurrent Neural Networks
• Variance Networks: When Expectation Does Not Meet Your Expectations
• A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference
• Weight Uncertainty in Neural Networks
• Learning Multiple Layers of Features from Tiny Images

Cited by

• ProbAct: A Probabilistic Activation Function for Deep Neural Networks
• On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
• Why have a Unified Predictive Uncertainty? Disentangling it using Deep\n Split Ensembles

Related

• Variational Neural Networks
• A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference
• Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty
• Variational Inference to Measure Model Uncertainty in Deep Neural Networks
• Projected BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weights