A Generalized Bias-Variance Decomposition for Bregman Divergences

2025/11/11 by David Pfau, Pfau, David · 1 voice · 2 citations
Physics and Astronomy · Computer Science · #Statistical Mechanics and Entropy #Stochastic Gradient Optimization Techniques #Gaussian Processes and Bayesian Inference

paper · pdf · doi:10.48550/arxiv.2511.08789

Abstract

The bias-variance decomposition is a central result in statistics and machine learning, but is typically presented only for the squared error. We present a generalization of the bias-variance decomposition where the prediction error is a Bregman divergence, which is relevant to maximum likelihood estimation with exponential families. While the result is already known, there was not previously a clear, standalone derivation, so we provide one for pedagogical purposes. A version of this note previously appeared on the author's personal website without context. Here we provide additional discussion and references to the relevant prior literature.

Cited by

• Subjective Risk Decomposition: A New View for Uncertainty Quantification
• Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
• When Does Predictive Inverse Dynamics Outperform Behavior Cloning?
• Bregman Stochastic Proximal Point Algorithm with Variance Reduction
• In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning

Discussions

• New (sort of) preprint on arXiv today: a generalized bias-variance decomposition for Bregman divergences! arxiv.org/abs/2511.08789 [bsky, 28 points, 1 comments]

Related

• Bias-variance decompositions: the exclusive privilege of Bregman divergences
• Understanding the bias-variance tradeoff of Bregman divergences
• The extended Bregman divergence and parametric estimation
• On minimum Bregman divergence inference
• Functional Bregman Divergence and Bayesian Estimation of Distributions