Using Lasso for Predictor Selection and to Assuage Overfitting: A Method Long Overlooked in Behavioral Sciences
2015/09/03 by Daniel McNeish, Daniel M. McNeish · 451 citations
Mathematics · #Advanced Causal Inference Techniques #Artificial intelligence #Artificial neural network #Computer science #Econometrics #Elastic net regularization #Feature selection #Generalizability theory #Lasso (programming language) #Machine learning #Mathematics #Model selection #Overfitting #Regression #Regularization (linguistics) #Selection (genetic algorithm) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics
paper · doi:10.1080/00273171.2015.1036965
published in Multivariate Behavioral Research 50(5), 471-484 (Taylor & Francis)
openalex publication_date 2015/09/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Abstract
Ordinary least squares and stepwise selection are widespread in behavioral science research; however, these methods are well known to encounter overfitting problems such that R(2) and regression coefficients may be inflated while standard errors and p values may be deflated, ultimately reducing both the parsimony of the model and the generalizability of conclusions. More optimal methods for selecting predictors and estimating regression coefficients such as regularization methods (e.g., Lasso) have existed for decades, are widely implemented in other disciplines, and are available in mainstream software, yet, these methods are essentially invisible in the behavioral science literature while the use of sub optimal methods continues to proliferate. This paper discusses potential issues with standard statistical models, provides an introduction to regularization with specific details on both Lasso and its related predecessor ridge regression, provides an example analysis and code for running a Lasso analysis in R and SAS, and discusses limitations and related methods.
Citations
Cited by
- Bayesian Estimation for Gaussian Graphical Models: Structure Learning, Predictability, and Network Comparisons
- A Practical Guide to Variable Selection in Structural Equation Modeling by Using Regularized Multiple-Indicators, Multiple-Causes Models
- Shrinkage priors for Bayesian penalized regression
- Exploratory Factor Analysis With Small Samples and Missing Data
- A Tutorial Introduction to Heterogeneous Treatment Effect Estimation with Meta-learners
- No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
- Machine learning for the life-time risk prediction of Alzheimer’s disease: a systematic review
- Enhancing measurement validity in diverse populations: Modern approaches to evaluating differential item functioning
- Why would people want to travel more with automated cars?
- Why is Regularization Underused? An Empirical Study on Trust and Adoption of Statistical Methods
- Regularized Structural Equation Modeling. [europepmc]
- Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. [europepmc]
- Exploratory Mediation Analysis via Regularization. [europepmc]
- Six-long non-coding RNA signature predicts recurrence-free survival in hepatocellular carcinoma. [europepmc]
- On Nonregularized Estimation of Psychological Networks. [europepmc]
- Back to the basics: Rethinking partial correlation network methodology. [europepmc]
- Premorbid and concurrent predictors of TMD onset and persistence. [europepmc]
- A Practical Guide to Variable Selection in Structural Equation Models with Regularized MIMIC Models. [europepmc]
- Predicting pathological complete response by comparing MRI-based radiomics pre- and postneoadjuvant radiotherapy for locally advanced rectal cancer. [europepmc]
- Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. [europepmc]
- Stress, Mindsets, and Success in Navy SEALs Special Warfare Training. [europepmc]
- Identification and Validation of a Prognostic lncRNA Signature for Hepatocellular Carcinoma. [europepmc]
- Predicting 5- and 10-Year Mortality Risk in Older Adults With Diabetes. [europepmc]
- Risk Prediction Models for Post-Operative Mortality in Patients With Cirrhosis. [europepmc]
- Machine learning and atherosclerotic cardiovascular disease risk prediction in a multi-ethnic population. [europepmc]
- Simplifying the Assessment of Measurement Invariance over Multiple Background Variables: Using Regularized Moderated Nonlinear Factor Analysis to Detect Differential Item Functioning. [europepmc]
- The impact of social distancing during COVID-19: A conditional process model of negative emotions, alienation, affective disorders, and post-traumatic stress disorder. [europepmc]
- Psychometric properties of the embodiment scale for the rubber hand illusion and its relation with individual differences. [europepmc]
- Lung adenocarcinoma and lung squamous cell carcinoma cancer classification, biomarker identification, and gene expression analysis using overlapping feature selection methods. [europepmc]
- Identifying Predictors of University Students' Wellbeing during the COVID-19 Pandemic-A Data-Driven Approach. [europepmc]
- Multiparametric Integrated 18 F-FDG PET/MRI-Based Radiomics for Breast Cancer Phenotyping and Tumor Decoding. [europepmc]
- AI-enhanced simultaneous multiparametric 18 F-FDG PET/MRI for accurate breast cancer diagnosis. [europepmc]
- Multiparametric 18 F-FDG PET/MRI-Based Radiomics for Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in Breast Cancer. [europepmc]
- Applying logistic LASSO regression for the diagnosis of atypical Crohn's disease. [europepmc]
- Biosensors, Artificial Intelligence Biosensors, False Results and Novel Future Perspectives. [europepmc]
Related