A Survey of Numerical Algorithms that can Solve the Lasso Problems

2023/03/07 by Yujie Zhao, Xiaoming Huo, Zhao, Yujie +1 · 2 citations
Mathematics · Medicine · #Computation (stat.CO) #FOS: Computer and information sciences #Machine Learning (stat.ML) #Statistical Methods and Inference #Systemic Lupus Erythematosus Research

paper · pdf · doi:10.48550/arxiv.2303.03576

openalex publication_date 2023/03/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

In statistics, the least absolute shrinkage and selection operator (Lasso) is a regression method that performs both variable selection and regularization. There is a lot of literature available, discussing the statistical properties of the regression coefficients estimated by the Lasso method. However, there lacks a comprehensive review discussing the algorithms to solve the optimization problem in Lasso. In this review, we summarize five representative algorithms to optimize the objective function in Lasso, including the iterative shrinkage threshold algorithm (ISTA), fast iterative shrinkage-thresholding algorithms (FISTA), coordinate gradient descent algorithm (CGDA), smooth L1 algorithm (SLA), and path following algorithm (PFA). Additionally, we also compare their convergence rate, as well as their potential strengths and weakness.

Cited by

• Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
• A fast algorithm for solving the lasso problem exactly without homotopy using differential inclusions

Related

• Lasso Regression: Estimation and Shrinkage via Limit of Gibbs Sampling
• A coordinate-wise optimization algorithm for the Fused Lasso
• Fast sparse optimization via adaptive shrinkage
• On an improvement of LASSO by scaling
• Lasso-Ridge Refitting: A Two-Stage Estimator for High-Dimensional Linear Regression