Scalable Bayesian neural networks by layer-wise input augmentation

2020/10/26 by Trung Trinh, Samuel Kaski, Trinh, Trung +3
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.2010.13498

openalex publication_date 2020/10/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce implicit Bayesian neural networks, a simple and scalable approach for uncertainty representation in deep learning. Standard Bayesian approach to deep learning requires the impractical inference of the posterior distribution over millions of parameters. Instead, we propose to induce a distribution that captures the uncertainty over neural networks by augmenting each layer's inputs with latent variables. We present appropriate input distributions and demonstrate state-of-the-art performance in terms of calibration, robustness and uncertainty characterisation over large-scale, multi-million parameter image classification tasks.

Citations

• Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs
• A Simple Baseline for Bayesian Uncertainty in Deep Learning
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Variational Inference: A Review for Statisticians

Related

• Implicit Weight Uncertainty in Neural Networks
• Scalable Bayesian Deep Learning with Kernel Seed Networks
• Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification
• Sparse Uncertainty Representation in Deep Learning with Inducing Weights
• Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure