Rethinking Collapsed Variational Bayes Inference for LDA

2012/06/27 by Sato, Issei, Nakagawa, Hiroshi
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1206.6435

Abstract

We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two different divergence projections: alpha=1 and -1. This interpretation will help shed light on CVB0 works.

Related

• Algorithms of the LDA model [REPORT]
• The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation
• Practical Collapsed Stochastic Variational Inference for the HDP
• Dense Distributions from Sparse Samples: Improved Gibbs Sampling Parameter Estimators for LDA
• Alpha-Divergences in Variational Dropout