Streamlining Prediction in Bayesian Deep Learning

2024/11/27 by Rui Li, Marcus Klasson, Li, Rui +5 · 3 voices · 1 citation
Computer Science · #Anomaly Detection Techniques and Applications #Artificial intelligence #Bayesian network #Bayesian probability #Computer science #Data Stream Mining Techniques #Deep learning #Gaussian Processes and Bayesian Inference #Machine learning #cs.LG

paper · pdf · doi:10.48550/arxiv.2411.18425

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/11/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

The rising interest in Bayesian deep learning (BDL) has led to a plethora of methods for estimating the posterior distribution. However, efficient computation of inferences, such as predictions, has been largely overlooked with Monte Carlo integration remaining the standard. In this work we examine streamlining prediction in BDL through a single forward pass without sampling. For this we use local linearisation on activation functions and local Gaussian approximations at linear layers. Thus allowing us to analytically compute an approximation to the posterior predictive distribution. We showcase our approach for both MLP and transformers, such as ViT and GPT-2, and assess its performance on regression and classification tasks. Open-source library: https://github.com/AaltoML/SUQ

Cited by

• Variational Visual Question Answering for Uncertainty-Aware Selective Prediction

Discussions

• We got an #ICLR paper 🎉! Awesome work done with @ruili-pml.bsky.social, @marcusklasson.bsky.social, and @arnosolin.bsky.social. More details will follow!!! arxiv.org/abs/2411.18425 [bsky, 27 points, 1 comments]
• 🎉 Presenting “Streamlining Prediction in Bayesian Deep Learning” at #ICLR2025! We replace MC estimation with local linearisation + Gaussian approximations → analytic posterior predictive in one forwa [bsky, 15 points, 2 comments]
• Our #ICLR2025 poster "Streamlining Prediction in Bayesian Deep Learning" (Rui Li · Marcus Klasson, Arno Solin, Martin Trapp) 🗓️ Hall 3 + Hall 2B #413, Fri 25 Apr 10 a.m. +08 — 12:30 p.m. +08 📄 Prepr [bsky, 11 points, 1 comments]

Related

• Improving predictions of Bayesian neural nets via local linearization
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization
• Bayes without Underfitting: Fully Correlated Deep Learning Posteriors via Alternating Projections
• Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distribution
• Bayesian Deep Learning via Subnetwork Inference