L₀ regularized estimation for nonlinear models that have sparse underlying linear structures

2009/10/14 by Zhiyi Chi, Chi, Zhiyi · 1 citation
Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Probability (math.PR) #Statistics Theory (math.ST) #math.PR #math.ST #stat.ML #stat.TH

paper · pdf · doi:10.48550/arxiv.0910.2517

arxiv created 2009/10/14 · arxiv updated 2009/12/01

Abstract

We study the estimation of β for the nonlinear model y = f(X\sp\topβ) + ε when f is a nonlinear transformation that is known, β has sparse nonzero coordinates, and the number of observations can be much smaller than that of parameters (n≪ p). We show that in order to bound the L₂ error of the L₀ regularized estimator β, i.e., ‖β- β‖₂, it is sufficient to establish two conditions. Based on this, we obtain bounds of the L₂ error for (1) L₀ regularized maximum likelihood estimation (MLE) for exponential linear models and (2) L₀ regularized least square (LS) regression for the more general case where f is analytic. For the analytic case, we rely on power series expansion of f, which requires taking into account the singularities of f.

Cited by

• On ℓ₁-regularized estimation for nonlinear models that have sparse underlying linear structures

Related

• On ℓ₁-regularized estimation for nonlinear models that have sparse underlying linear structures
• Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
• Sparse Recovery via Partial Regularization: Models, Theory and Algorithms
• Sparsistency of ℓ₁-Regularized M-Estimators
• Simple Error Bounds for Regularized Noisy Linear Inverse Problems