Finite mixture regression: A sparse variable selection by model selection for clustering

2014/09/04 by Émilie Devijver, Devijver, Emilie
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Mathematics #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1409.1331

openalex publication_date 2014/09/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider a finite mixture of Gaussian regression model for high- dimensional data, where the number of covariates may be much larger than the sample size. We propose to estimate the unknown conditional mixture density by a maximum likelihood estimator, restricted on relevant variables selected by an 1-penalized maximum likelihood estimator. We get an oracle inequality satisfied by this estimator with a Jensen-Kullback-Leibler type loss. Our oracle inequality is deduced from a general model selection theorem for maximum likelihood estimators with a random model collection. We can derive the penalty shape of the criterion, which depends on the complexity of the random model collection.

Related

• Full Model Estimation for Non-Parametric Multivariate Finite Mixture Models
• From here to infinity - sparse finite versus Dirichlet process mixtures in model-based clustering
• Clustering and Model Selection via Penalized Likelihood for Different-sized Categorical Data Vectors
• Conditional Density Estimation by Penalized Likelihood Model Selection and Applications
• Model Selection for Gaussian Mixture Models