On Stein's Identity and Near-Optimal Estimation in High-dimensional Index Models

2017/09/26 by Zhuoran Yang, Krishnakumar Balasubramanian, Yang, Zhuoran +3
Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Methodology (stat.ME) #Random Matrices and Applications #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1709.08795

openalex publication_date 2017/09/26 · openalex created_date 2017/10/06 · openalex updated_date 2026/07/28

Abstract

We consider estimating the parametric components of semi-parametric multiple index models in a high-dimensional and non-Gaussian setting. Such models form a rich class of non-linear models with applications to signal processing, machine learning and statistics. Our estimators leverage the score function based first and second-order Stein's identities and do not require the covariates to satisfy Gaussian or elliptical symmetry assumptions common in the literature. Moreover, to handle score functions and responses that are heavy-tailed, our estimators are constructed via carefully thresholding their empirical counterparts. We show that our estimator achieves near-optimal statistical rate of convergence in several settings. We supplement our theoretical results via simulation experiments that confirm the theory.

Citations

• Score Function Features for Discriminative Learning: Matrix and Tensor Framework
• The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties
• Structured signal recovery from non-linear and heavy-tailed measurements

Related

• High-dimensional Varying Index Coefficient Models via Stein's Identity
• High-dimensional Index Volatility Models via Stein's Identity
• High-Dimensional Single-Index Models: Link Estimation and Marginal Inference
• Optimal Smoothing in Single-Index Models
• Estimating multi-index models with response-conditional least squares