2019/08/10 by Yunan Wu, Lan Wang, Wu, Yunan +1 · 2 citations
Computer Science · Engineering · Mathematics · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #cs.LG #stat.ME #stat.ML
paper · pdf · doi:10.48550/arxiv.1908.03669
28 pages, 2 figures
arxiv created 2019/08/10 · openalex publication_date 2019/08/10 · arxiv updated 2019/08/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which determines the amount of regularization and hence the sparsity level of the fitted model. The optimal choice of tuning parameter depends on both the structure of the design matrix and the unknown random error distribution (variance, tail behavior, etc). This article reviews the current literature of tuning parameter selection for high-dimensional regression from both theoretical and practical perspectives. We discuss various strategies that choose the tuning parameter to achieve prediction accuracy or support recovery. We also review several recently proposed methods for tuning-free high-dimensional regression.