2020/02/18 by A. E. Allahverdyan, Allahverdyan, A. E.
Computer Science · Mathematics · Physics and Astronomy · #Bayesian Methods and Mixture Models #Data Analysis #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Distribution Estimation and Applications #Statistical Mechanics (cond-mat.stat-mech) #Statistical Mechanics and Entropy #Statistics and Probability (physics.data-an) #cond-mat.stat-mech #cs.LG #physics.data-an #stat.ML
paper · pdf · doi:10.48550/arxiv.2002.07884
25 pages, 1 figure
arxiv created 2020/02/18 · openalex publication_date 2020/02/18 · arxiv updated 2020/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We study the parameter estimation problem in mixture models with observational nonidentifiability: the full model (also containing hidden variables) is identifiable, but the marginal (observed) model is not. Hence global maxima of the marginal likelihood are (infinitely) degenerate and predictions of the marginal likelihood are not unique. We show how to generalize the marginal likelihood by introducing an effective temperature, and making it similar to the free energy. This generalization resolves the observational nonidentifiability, since its maximization leads to unique results that are better than a random selection of one degenerate maximum of the marginal likelihood or the averaging over many such maxima. The generalized likelihood inherits many features from the usual likelihood, e.g. it holds the conditionality principle, and its local maximum can be searched for via suitably modified expectation-maximization method. The maximization of the generalized likelihood relates to entropy optimization.