Variational Neural Networks

2022/07/04 by Oleksiienko, Illia, Tran, Dat Thanh, Iosifidis, Alexandros
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2207.01524

Abstract

Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other single-bin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods.

Related

• Can Bayesian Neural Networks Explicitly Model Input Uncertainty?
• Known Unknowns: Uncertainty Quality in Bayesian Neural Networks
• Variational Inference on the Final-Layer Output of Neural Networks
• Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertainty
• Projected BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weights