Learning mixtures of spherical Gaussians: moment methods and spectral decompositions

2012/06/25 by Daniel Hsu, Sham M. Kakade, Hsu, Daniel +1 · 11 citations
Chemistry · Computer Science · Mathematics · #Bayesian Methods and Mixture Models #Blind Source Separation Techniques #Spectroscopy and Chemometric Analyses #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1206.5766

arxiv created 2012/10/28 · arxiv updated 2012/10/30

Abstract

This work provides a computationally efficient and statistically consistent moment-based estimator for mixtures of spherical Gaussians. Under the condition that component means are in general position, a simple spectral decomposition technique yields consistent parameter estimates from low-order observable moments, without additional minimum separation assumptions needed by previous computationally efficient estimation procedures. Thus computational and information-theoretic barriers to efficient estimation in mixture models are precluded when the mixture components have means in general position and spherical covariances. Some connections are made to estimation problems related to independent component analysis.

Citations

• Independent component analysis: algorithms and applications

Cited by

• Efficient Topic Model Estimation under Heavy-Tailed Document Lengths
• Solving exact and noisy rank-one tensor completion with semidefinite programming
• Generalized Additive Decompositions of Symmetric Tensors
• Polynomial methods in statistical inference: theory and practice
• Whitening Spherical Gaussian Mixtures in the Large-Dimensional Regime
• Coupled tensor models for probability mass function estimation: Part I, Principles and algorithms
• Diagonally-Weighted Generalized Method of Moments Estimation for Gaussian Mixture Modeling
• Private Training & Data Generation by Clustering Embeddings
• Provable Bounds for Learning Some Deep Representations
• Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixtures
• Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures

Related

• A Fourier Approach to Mixture Learning
• Learning Gaussian Mixtures with Arbitrary Separation
• List-Decodable Robust Mean Estimation and Learning Mixtures of Spherical Gaussians
• Near-optimal-sample estimators for spherical Gaussian mixtures
• Mixture Models, Robustness, and Sum of Squares Proofs