Fast Variational Inference in the Conjugate Exponential Family

2012/06/22 by James Hensman, Hensman, James, Magnus Rattray +3 · 6 citations
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1206.5162

openalex publication_date 2012/06/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our collapsed variational inference leads to a new lower bound on the marginal likelihood. We exploit the information geometry of the bound to derive much faster optimization methods based on conjugate gradients for these models. Our approach is very general and is easily applied to any model where the mean field update equations have been derived. Empirically we show significant speed-ups for probabilistic models optimized using our bound.

Cited by

• Natural Gradient VI: Guarantees for Non-Conjugate Models
• Improving predictions of Bayesian neural nets via local linearization
• Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations
• Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
• Myopic Bayesian Design of Experiments via Posterior Sampling and Probabilistic Programming
• Fast and accurate approximate inference of transcript expression from\n RNA-seq data

Related

• Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models
• On variational inference and maximum likelihood estimation with the λ-exponential family
• Variational Inference: A Review for Statisticians
• Variational Inference: A Review for Statisticians
• A Generalized Mean Field Algorithm for Variational Inference in Exponential Families