Guidelines for LASSO and derivatives use under different dependence and scale structures

2025/06/10 by Laura Freijeiro‐González, Freijeiro-González, Laura, Manuel Febrero–Bande +3
Mathematics · #Statistical Methods and Inference #Advanced Statistical Methods and Models #Statistical Methods and Bayesian Inference

paper · pdf · doi:10.48550/arxiv.2506.08582

Abstract

In a multivariate linear regression model with p>1 covariates, implementation of penalization techniques often implies a preliminary univariate standardization step. Although this prevents scale effects on the covariates selection procedure, possible dependence structures can be disrupted, leading to wrong results. This is particularly challenging in high-dimensional settings where p ≥ n. In this paper, we analyze the standardization effect on the LASSO for different dependence-scales contexts by means of an extensive simulation study. Two distinct objectives are pursued: adequate covariate selection and proper predictive capability. Additionally, its behavior is compared with the one of some well-known or innovative competitors. This comparison is also extended to three real datasets facing different dependence-scales patterns. Eventually, we conclude with discussion and guidelines on the most suitable methodology for each case in terms of covariates selection or prediction.

Citations

• A critical review of LASSO and its derivatives for variable selection\n under dependence among covariates
• The Generalized Lasso Problem and Uniqueness
• Variable selection in Functional Additive Regression Models
• False Discoveries Occur Early on the Lasso Path
• Statistical Learning with Sparsity: The Lasso and Generalizations
• High-Dimensional Inference: Confidence Intervals, p-Values and R-Software hdi
• High-dimensional regression with unknown variance
• Scaled Sparse Linear Regression
• Modeling wine preferences by data mining from physicochemical properties
• Stability Selection
• Measuring and testing dependence by correlation of distances
• The Dantzig selector: Statistical estimation when p is much larger than n
• Greedy function approximation: A gradient boosting machine.
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Applied Regression Analysis

Related

• A critical review of LASSO and its derivatives for variable selection\n under dependence among covariates
• Asymptotic Theory of K-fold Cross-validation in Lasso and the validity of Bootstrap
• New Error Analysis for Lasso
• Comparisons of penalized least squares methods by simulations
• Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models