Variable Selection with Exponential Weights and l₀-Penalization

2012/08/13 by Ery Arias-Castro, Arias-Castro, Ery, Karim Lounici +1
Engineering · Mathematics · #Advanced Statistical Methods and Models #Control Systems and Identification #FOS: Mathematics #Statistical Methods and Inference #Statistics Theory (math.ST) #math.ST #stat.TH

paper · pdf · doi:10.48550/arxiv.1208.2635

23 pages; 1 figures

openalex publication_date 2012/08/13 · arxiv created 2012/09/17 · arxiv updated 2012/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In the context of a linear model with a sparse coefficient vector, exponential weights methods have been shown to be achieve oracle inequalities for prediction. We show that such methods also succeed at variable selection and estimation under the necessary identifiability condition on the design matrix, instead of much stronger assumptions required by other methods such as the Lasso or the Dantzig Selector. The same analysis yields consistency results for Bayesian methods and BIC-type variable selection under similar conditions.

Citations

• Nearly unbiased variable selection under minimax concave penalty

Related

• On the conditions used to prove oracle results for the Lasso
• Thresholded Lasso for high dimensional variable selection
• Thresholded Lasso for high dimensional variable selection and statistical estimation
• Combined l₁ and greedy l₀ penalized least squares for linear model selection
• Sparse Oracle Inequalities for Variable Selection via Regularized Quantization