Iteratively Reweighted ℓ₁-Penalized Robust Regression

2019/07/09 by Xiaoou Pan, Qiang Sun, Pan, Xiaoou +3
Decision Sciences · Engineering · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Risk and Portfolio Optimization #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1907.04027

openalex publication_date 2019/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted ℓ₁-penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as O(log s + loglog d) iterations to reach such an oracle estimator, where s and d denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

Citations

• Nearly unbiased variable selection under minimax concave penalty
• Optimization Transfer Using Surrogate Objective Functions
• Robust Estimation of a Location Parameter
• Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression

Related

• Statistical consistency and asymptotic normality for high-dimensional\n robust M-estimators
• Retire: Robust Expectile Regression in High Dimensions
• Adaptive Huber Regression
• Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise
• Scale calibration for high-dimensional robust regression