Sequential Local Learning for Latent Graphical Models

2017/03/12 by Park, Sejun, Yang, Eunho, Shin, Jinwoo
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1703.04082

Abstract

Learning parameters of latent graphical models (GM) is inherently much harder than that of no-latent ones since the latent variables make the corresponding log-likelihood non-concave. Nevertheless, expectation-maximization schemes are popularly used in practice, but they are typically stuck in local optima. In the recent years, the method of moments have provided a refreshing angle for resolving the non-convex issue, but it is applicable to a quite limited class of latent GMs. In this paper, we aim for enhancing its power via enlarging such a class of latent GMs. To this end, we introduce two novel concepts, coined marginalization and conditioning, which can reduce the problem of learning a larger GM to that of a smaller one. More importantly, they lead to a sequential learning framework that repeatedly increases the learning portion of given latent GM, and thus covers a significantly broader and more complicated class of loopy latent GMs which include convolutional and random regular models.

Related

• Fast Algorithms for Learning Latent Variables in Graphical Models
• Learning Exponential Family Graphical Models with Latent Variables using\n Regularized Conditional Likelihood
• Learning General Latent-Variable Graphical Models with Predictive Belief Propagation
• Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics
• On Learning Discrete Graphical Models Using Greedy Methods