Streaming Variational Bayes

2013/07/25 by Tamara Broderick, Nicholas Boyd, Broderick, Tamara +7 · 9 citations
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1307.6769

openalex publication_date 2013/07/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) as the primitive, by fitting the latent Dirichlet allocation model to two large-scale document collections. We demonstrate the advantages of our algorithm over stochastic variational inference (SVI) by comparing the two after a single pass through a known amount of data---a case where SVI may be applied---and in the streaming setting, where SVI does not apply.

Citations

• The Big Data Bootstrap
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

Cited by

• Unlearning in Diffusion models under Data Constraints: A Variational Inference Approach
• Variational Federated Multi-Task Learning
• Online Coreset Selection for Rehearsal-based Continual Learning
• Quasi-Bayes properties of a recursive procedure for mixtures
• MLego: Interactive and Scalable Topic Exploration Through Model Reuse
• Probabilistic Line Searches for Stochastic Optimization
• Low Complexity Approximate Bayesian Logistic Regression for Sparse Online Learning
• Stochastic Variational Inference for Hidden Markov Models
• Streaming, Distributed Variational Inference for Bayesian Nonparametrics

Related

• The Population Posterior and Bayesian Inference on Streams
• Streaming Bayes GFlowNets
• Stochastic Variational Inference
• Streaming, Distributed Variational Inference for Bayesian Nonparametrics
• Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet\n Allocation