2007/07/01 by Saharon Rosset, Ji Zhu · 4 citations
Engineering · Mathematics · #Advanced Statistical Methods and Models #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #math.ST #msc:62F35 #msc:62G08 #msc:62H30 #msc:62J07 #stat.ML #stat.TH
paper · pdf · doi:10.1214/009053606000001370
published as Annals of Statistics 2007, Vol. 35, No. 3, 1012-1030 · Published at http://dx.doi.org/10.1214/009053606000001370 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
openalex publication_date 2007/07/01 · arxiv created 2007/08/16 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31
We consider the generic regularized optimization problem β̂(λ)=arg minβ L(y, Xβ)+λJ(β). Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407–499] have shown that for the LASSO—that is, if L is squared error loss and J(β)=‖β‖1 is the ℓ1 norm of β—the optimal coefficient path is piecewise linear, that is, ∂β̂(λ)/∂λ is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the LASSO for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen and van de Geer’s locally adaptive regression splines.