Exact post-selection inference, with application to the lasso

2013/11/30 by Jason D. Lee, Dennis L. Sun, Yuekai Sun +1 · 6 citations
Mathematics · #Advanced Statistical Methods and Models #Core (optical fiber) #Estimator #Feature selection #Inference #Lasso (programming language) #Model selection #Selection (genetic algorithm) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistical hypothesis testing #math.ST #stat.ME #stat.ML #stat.TH

paper · pdf · doi:10.1214/15-aos1371

published as Annals of Statistics 2016, Vol. 44, No. 3, 907-927 · Published at http://dx.doi.org/10.1214/15-AOS1371 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2016/04/11 · arxiv created 2016/05/03 · arxiv updated 2016/05/04 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/06

Abstract

We develop a general approach to valid inference after model selection. At the core of our framework is a result that characterizes the distribution of a post-selection estimator conditioned on the selection event. We specialize the approach to model selection by the lasso to form valid confidence intervals for the selected coefficients and test whether all relevant variables have been included in the model.

Citations

• Least angle regression
• Can one estimate the conditional distribution of post-model-selection estimators?
• The positive false discovery rate: a Bayesian interpretation and the q-value

Cited by

• Post Selection Estimation of Sharpe Ratios
• Spurious Predictability in Financial Machine Learning
• Statistical Inference for Clustering-based Anomaly Detection
• Post-Transfer Learning Statistical Inference in High-Dimensional Regression
• Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selection
• Soprano voices in opera seria: a corpus-based inquiry into eighteenth-century vocal types
• Optimal post-selection inference for sparse signals: a nonparametric empirical Bayes approach