Sparse Stochastic Inference for Latent Dirichlet allocation

2012/06/27 by David Mimno, Matt Hoffman, Mimno, David +3
Computer Science · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1206.6425

Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012)

arxiv created 2012/06/27 · arxiv updated 2012/07/02

Abstract

We present a hybrid algorithm for Bayesian topic models that combines the efficiency of sparse Gibbs sampling with the scalability of online stochastic inference. We used our algorithm to analyze a corpus of 1.2 million books (33 billion words) with thousands of topics. Our approach reduces the bias of variational inference and generalizes to many Bayesian hidden-variable models.

Related

• Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation
• Stochastic Variational Inference
• On Smoothing and Inference for Topic Models
• Inference in topic models: sparsity and trade-off
• Sparse Partially Collapsed MCMC for Parallel Inference in Topic Models