Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference

2015/11/14 by Zhuoran Yang, Zhaoran Wang, Yang, Zhuoran +7 · 1 citation
Engineering · Mathematics · #Advanced Statistical Methods and Models #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.1511.04514

openalex publication_date 2015/11/14 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by y = f( x^\top β^* ) + ε, where f is nonlinear. To recover β^*, we propose an ℓ₁-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f. In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. In addition, we provide an efficient algorithm that provably converges to a stationary point. We also access the uncertainty of the obtained estimator. Specifically, based on any stationary point of the objective, we construct valid hypothesis tests and confidence intervals for the low dimensional components of the high-dimensional parameter β^*. Detailed numerical results are provided to back up our theory.

Citations

• A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models
• Low-rank Solutions of Linear Matrix Equations via Procrustes Flow
• Sparse Recovery with Linear and Nonlinear Observations: Dependent and Noisy Data
• Optimal Smoothing in Single-Index Models
• Phase Retrieval with Application to Optical Imaging: A contemporary overview
• PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming
• Efficient Learning of Generalized Linear and Single Index Models with\n Isotonic Regression
• Exact Post-Selection Inference for Sequential Regression Procedures
• A Likelihood Ratio Framework for High Dimensional Semiparametric Regression
• The generalized Lasso with non-linear observations
• Exponential Screening and optimal rates of sparse estimation

Cited by

• Misspecified Nonconvex Statistical Optimization for Phase Retrieval

Related

• SLOPE for Sparse Linear Regression:Asymptotics and Optimal Regularization
• L₀ regularized estimation for nonlinear models that have sparse underlying linear structures
• Sparse recovery via nonconvex regularized M-estimators over ℓ_q-balls
• Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression
• Implicit Regularization for Optimal Sparse Recovery