2015/06/25 by Reshad Hosseini, Suvrit Sra, Hosseini, Reshad +1
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.1506.07677
openalex publication_date 2015/06/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We take a new look at parameter estimation for Gaussian Mixture Models (GMMs). In particular, we propose using Riemannian manifold optimization as a powerful counterpart to Expectation Maximization (EM). An out-of-the-box invocation of manifold optimization, however, fails spectacularly: it converges to the same solution but vastly slower. Driven by intuition from manifold convexity, we then propose a reparamerization that has remarkable empirical consequences. It makes manifold optimization not only match EM---a highly encouraging result in itself given the poor record nonlinear programming methods have had against EM so far---but also outperform EM in many practical settings, while displaying much less variability in running times. We further highlight the strengths of manifold optimization by developing a somewhat tuned manifold LBFGS method that proves even more competitive and reliable than existing manifold optimization tools. We hope that our results encourage a wider consideration of manifold optimization for parameter estimation problems.