A Geometric View on Constrained M-Estimators

2015/06/26 by Yen-Huan Li, Li, Yen-Huan, Ya-Ping Hsieh +5 · 1 citation
Mathematics · #Advanced Statistical Methods and Models #FOS: Mathematics #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1506.08163

openalex publication_date 2015/06/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the estimation error of constrained M-estimators, and derive explicit upper bounds on the expected estimation error determined by the Gaussian width of the constraint set. Both of the cases where the true parameter is on the boundary of the constraint set (matched constraint), and where the true parameter is strictly in the constraint set (mismatched constraint) are considered. For both cases, we derive novel universal estimation error bounds for regression in a generalized linear model with the canonical link function. Our error bound for the mismatched constraint case is minimax optimal in terms of its dependence on the sample size, for Gaussian linear regression by the Lasso.

Citations

• Simple Bounds for Noisy Linear Inverse Problems with Exact Side Information
• The generalized Lasso with non-linear observations

Cited by

• Asymptotics of constrained M-estimation under convexity

Related

• Variational Analysis of Constrained M-Estimators
• Generalized Linear Model Regression under Distance-to-set Penalties
• The Mismatch Principle: The Generalized Lasso Under Large Model Uncertainties
• Mean Square Error bounds for parameter estimation under model misspecification
• Statistical inference for high dimensional regression via Constrained Lasso