On the adaptive elastic-net with a diverging number of parameters

2009/06/18 by Hui Zou, Hao Helen Zhang · 874 citations
Engineering · Mathematics · #3D Shape Modeling and Analysis #Advanced Numerical Analysis Techniques #Applied mathematics #Econometrics #Elastic net regularization #Geometry #Mathematics #Morphological variations and asymmetry #Net (polyhedron) #Regression #Statistical physics #Statistics #math.ST #msc:62J05 #msc:62J07 #stat.TH

paper · pdf · doi:10.1214/08-aos625

published in The Annals of Statistics 37(4), 1733-1751 (Institute of Mathematical Statistics) · Published in at http://dx.doi.org/10.1214/08-AOS625 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2009/06/18 · arxiv created 2009/08/13 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We consider the problem of model selection and estimation in situations where the number of parameters diverges with the sample size. When the dimension is high, an ideal method should have the oracle property (Fan and Li, 2001; Fan and Peng, 2004) which ensures the optimal large sample performance. Furthermore, the high-dimensionality often induces the collinearity problem which should be properly handled by the ideal method. Many existing variable selection methods fail to achieve both goals simultaneously. In this paper, we propose the adaptive Elastic-Net that combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. Under weak regularity conditions, we establish the oracle property of the adaptive Elastic-Net. We show by simulations that the adaptive Elastic-Net deals with the collinearity problem better than the other oracle-like methods, thus enjoying much improved finite sample performance.

Citations

• Least angle regression
• Ideal spatial adaptation by wavelet shrinkage

Cited by

• Marked point processes intensity estimation using sparse group Lasso method applied to locations of lucrative and cooperative banks in mainland France
• Adaptive Elastic Net Method for Cox Model
• Elastic Net Procedure for Partially Linear Models
• Survival Modeling of Suicide Risk with Rare and Uncertain Diagnoses
• Sparse Distance Weighted Discrimination
• Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension
• Regularization techniques for inhomogeneous (spatial) point processes intensity and conditional intensity estimation
• Reluctant Transfer Learning in Penalized Regressions for Individualized Treatment Rules under Effect Heterogeneity
• Health Analytics: a systematic review of approaches to detect phenotype cohorts using electronic health records
• Data Mining for Longitudinal Data under Multicollinearity and Time Dependence using Penalized Generalized Estimating Equations
• Spatial point processes intensity estimation with a diverging number of covariates
• The oracle property of the generalized outcome-adaptive lasso
• Fixed Effects and the Generalized Mundlak Estimator
• Fast and Scalable Lasso via Stochastic Frank-Wolfe Methods with a Convergence Guarantee
• An adaptive subsampling method for large-sample feature screening
• Spatial Variable Selection and An Application to Virginia Lyme Disease Emergence
• Variable Selection with Multiply-Imputed Datasets: Choosing Between Stacked and Grouped Methods
• MIBoost: A Gradient Boosting Algorithm for Variable Selection After Multiple Imputation
• Greedy Forward Regression for Variable Screening
• Variable Selection for Survival Data with A Class of Adaptive Elastic Net Techniques
• Sufficiency-principled Transfer Learning via Model Averaging
• Binary Fused Compressive Sensing: 1-Bit Compressive Sensing meets Group Sparsity
• Parameter-transfer in spatial autoregressive models via model averaging
• Variable selection in high-dimensional linear model with possibly asymmetric or heavy-tailed errors
• Least Squares Approximation for a Distributed System
• Endogenous Treatment Effect Estimation with some Invalid and Irrelevant Instruments
• Machine Learning Advances for Time Series Forecasting
• Integrated conditional moment test and beyond: when the number of covariates is divergent
• Adaptive elastic-net selection in a quantile model with diverging number of variable groups
• Regularization for supervised learning via the "hubNet" procedure
• Adaptive elastic net and Separate Selection from Least Squares for ultra-high dimensional regression models
• Questioning the news about economic growth: Sparse forecasting using thousands of news-based sentiment values
• Generalized Co-sparse Factor Regression
• Penalty, Shrinkage, and Preliminary Test Estimators under Full Model Hypothesis
• The BerHu penalty and the grouped effect
• Aligning Language Models with Observational Data: Opportunities and Risks from a Causal Perspective
• Grouped Variable Selection via Nested Spike and Slab Priors
• A Network-Guided Penalized Regression with Application to Proteomics Data
• Elastic Net based Feature Ranking and Selection
• Reconstructing large networks with time-varying interactions
• The Smooth-Lasso and other ℓ₁+ℓ₂-penalized methods
• Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• On Hodges’ superefficiency and merits of oracle property in model selection
• GEE analysis of clustered binary data with diverging number of covariates

Related

• The adaptive Gril estimator with a diverging number of parameters
• Prediction Weighted Maximum Frequency Selection
• Adaptive elastic net and Separate Selection from Least Squares for ultra-high dimensional regression models
• Variable selection and regularization via arbitrary rectangle-range generalized elastic net
• Variable Selection and Regularization via Arbitrary Rectangle-range Generalized Elastic Net