2023/12/20 by Pierre Bellec, Bellec, Pierre C., Takuya Koriyama +1
Mathematics · #Advanced Statistical Methods and Models #FOS: Mathematics #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2312.13257
openalex publication_date 2023/12/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider unregularized robust M-estimators for linear models under Gaussian design and heavy-tailed noise, in the proportional asymptotics regime where the sample size n and the number of features p are both increasing such that p/n → γ∈ (0,1). An estimator of the out-of-sample error of a robust M-estimator is analyzed and proved to be consistent for a large family of loss functions that includes the Huber loss. As an application of this result, we propose an adaptive tuning procedure of the scale parameter λ>0 of a given loss function ρ: choosing λ in a given interval I that minimizes the out-of-sample error estimate of the M-estimator constructed with loss ρλ(⋅) = λ2 ρ(⋅/λ) leads to the optimal out-of-sample error over I. The proof relies on a smoothing argument: the unregularized M-estimation objective function is perturbed, or smoothed, with a Ridge penalty that vanishes as n→+∞, and shows that the unregularized M-estimator of interest inherits properties of its smoothed version.