One-step sparse estimates in nonconcave penalized likelihood models

2008/07/16 by Hui Zou, Runze Li · 14 citations
Computer Science · Mathematics · #Advanced Statistical Methods and Models #Machine Learning and Algorithms #Statistical Methods and Inference #math.ST #msc:62J05 #msc:62J07 #stat.TH

paper · pdf · doi:10.1214/009053607000000802

published as Annals of Statistics 2008, Vol. 36, No. 4, 1509-1533 · This paper discussed in: [arXiv:0808.1013], [arXiv:0808.1016], [arXiv:0808.1025]. Rejoinder in [arXiv:0808.1030]. Published in at http://dx.doi.org/10.1214/009053607000000802 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2008/07/16 · arxiv created 2008/08/07 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/01

Abstract

Fan & Li (2001) propose a family of variable selection methods via penalized likelihood using concave penalty functions. The nonconcave penalized likelihood estimators enjoy the oracle properties, but maximizing the penalized likelihood function is computationally challenging, because the objective function is nondifferentiable and nonconcave. In this article we propose a new unified algorithm based on the local linear approximation (LLA) for maximizing the penalized likelihood for a broad class of concave penalty functions. Convergence and other theoretical properties of the LLA algorithm are established. A distinguished feature of the LLA algorithm is that at each LLA step, the LLA estimator can naturally adopt a sparse representation. Thus we suggest using the one-step LLA estimator from the LLA algorithm as the final estimates. Statistically, we show that if the regularization parameter is appropriately chosen, the one-step LLA estimates enjoy the oracle properties with good initial estimators. Computationally, the one-step LLA estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood. We conduct some Monte Carlo simulation to assess the finite sample performance of the one-step sparse estimation methods. The results are very encouraging.

Citations

• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models: Who cares if it is a white cat or a black cat?
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Rejoinder: One-step sparse estimates in nonconcave penalized likelihood models
• Statistical Challenges with High Dimensionality: Feature Selection in Knowledge Discovery
• Nonconcave penalized likelihood with a diverging number of parameters
• Local partial-likelihood estimation for lifetime data
• Variable selection in semiparametric regression modeling
• Piecewise linear regularized solution paths
• Least angle regression
• A Statistical View of Some Chemometrics Regression Tools

Cited by

• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models: Who cares if it is a white cat or a black cat?
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models
• Rejoinder: One-step sparse estimates in nonconcave penalized likelihood models
• Non‐convex penalized multitask regression using data depth‐based penalties
• Robust rank correlation based screening
• Collaborative Inference for Sparse High-Dimensional Models with Non-Shared Data
• Least angle and ℓ1 penalized regression: A review
• Smoothing ℓ1-penalized estimators for high-dimensional time-course data
• Smooth SCAD: A Raised Cosine SCAD Type Thresholding Rule for Wavelet Denoising
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• Selection of tuning parameters in bridge regression models via Bayesian information criterion
• A fast algorithm for detecting gene–gene interactions in genome-wide association studies
• Functional response additive model estimation with online virtual stock markets
• A Model-free Variable Screening Method Based on Leverage Score
• Finite Mixture Cox Model for Heterogeneous Time-dependent Right-Censored Data

Related

• Nonconcave penalized likelihood with a diverging number of parameters
• Strong oracle optimality of folded concave penalized estimation
• Non-Concave Penalized Likelihood with NP-Dimensionality
• Sparse estimation via nonconcave penalized likelihood in a factor analysis model
• Sorted Concave Penalized Regression