Particle algorithms for maximum likelihood training of latent variable models

2022/04/27 by Juan Kuntz, Kuntz, Juan, Lim, Jen Ning +1 · 5 citations
Computer Science · #Bayesian Methods and Mixture Models #Computation (stat.CO) #Data Analysis with R #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2204.12965

openalex publication_date 2022/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

(Neal and Hinton, 1998) recast maximum likelihood estimation of any given latent variable model as the minimization of a free energy functional F, and the EM algorithm as coordinate descent applied to F. Here, we explore alternative ways to optimize the functional. In particular, we identify various gradient flows associated with F and show that their limits coincide with F's stationary points. By discretizing the flows, we obtain practical particle-based algorithms for maximum likelihood estimation in broad classes of latent variable models. The novel algorithms scale to high-dimensional settings and perform well in numerical experiments.

Cited by

• Particle Dynamics for Latent-Variable Energy-Based Models
• Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics
• A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models
• Divide, Interact, Sample: The Two-System Paradigm
• Modelling birdsong transmission with methods from molecular sequence analysis

Related

• Momentum Particle Maximum Likelihood
• Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Betting
• Particle Dynamics for Latent-Variable Energy-Based Models
• Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics
• Training Latent Diffusion Models with Interacting Particle Algorithms