Post-hoc loss-calibration for Bayesian neural networks

2021/06/13 by Meet P. Vadera, Vadera, Meet P., Soumya Ghosh +5
Computer Science · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2106.06997

openalex publication_date 2021/06/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posteriors that are approximated with, potentially crude, surrogates. This difficulty has engendered loss-calibrated techniques that aim to learn posterior approximations that favor high-utility decisions. In this paper, focusing on Bayesian neural networks, we develop methods for correcting approximate posterior predictive distributions encouraging them to prefer high-utility decisions. In contrast to previous work, our approach is agnostic to the choice of the approximate inference algorithm, allows for efficient test time decision making through amortization, and empirically produces higher quality decisions. We demonstrate the effectiveness of our approach through controlled experiments spanning a diversity of tasks and datasets.

Related

• Loss-Calibrated Approximate Inference in Bayesian Neural Networks
• Understanding Approximation for Bayesian Inference in Neural Networks
• Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks
• Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networks
• Loss-calibrated expectation propagation for approximate Bayesian decision-making