2018/01/01 by Badr-Eddine Chérief-Abdellatif, Pierre Alquier
Computer Science · Mathematics · #Applied mathematics #Artificial intelligence #Bayes' theorem #Bayesian Methods and Mixture Models #Bayesian inference #Bayesian probability #Computer science #Consistency (knowledge bases) #Convergence (economics) #Inference #Machine learning #Mathematical optimization #Mathematics #Model selection #Selection (genetic algorithm) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics #math.ST #stat.CO #stat.ME #stat.TH
paper · pdf · doi:10.1214/18-ejs1475
published as Electronic Journal of Statistics, 2018, vol. 12, no. 2, pp. 2995-3035
openalex publication_date 2018/01/01 · openalex created_date 2018/05/17 · arxiv created 2018/08/12 · arxiv updated 2020/08/03 · openalex updated_date 2026/08/06
Mixture models are widely used in Bayesian statistics and machine learning, in particular in computational biology, natural language processing and many other fields. Variational inference, a technique for approximating intractable posteriors thanks to optimization algorithms, is extremely popular in practice when dealing with complex models such as mixtures. The contribution of this paper is two-fold. First, we study the concentration of variational approximations of posteriors, which is still an open problem for general mixtures, and we derive consistency and rates of convergence. We also tackle the problem of model selection for the number of components: we study the approach already used in practice, which consists in maximizing a numerical criterion (the Evidence Lower Bound). We prove that this strategy indeed leads to strong oracle inequalities. We illustrate our theoretical results by applications to Gaussian and multinomial mixtures.