2015/11/25 by Matey Neykov, Jun S. Liu, Neykov, Matey +3 · 2 citations
Computer Science · Mathematics · #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #FOS: Mathematics #Geochemistry and Geologic Mapping #Machine Learning (stat.ML) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1511.08102
openalex publication_date 2015/11/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
It is known that for a certain class of single index models (SIMs) Y = f(\boldsymbolXp × 1^\intercal\boldsymbolβ0, ε), support recovery is impossible when \boldsymbolX ∼ N(0, \mathbbIp × p) and a model complexity adjusted sample size is below a critical threshold. Recently, optimal algorithms based on Sliced Inverse Regression (SIR) were suggested. These algorithms work provably under the assumption that the design \boldsymbolX comes from an i.i.d. Gaussian distribution. In the present paper we analyze algorithms based on covariance screening and least squares with L1 penalization (i.e. LASSO) and demonstrate that they can also enjoy optimal (up to a scalar) rescaled sample size in terms of support recovery, albeit under slightly different assumptions on f and ε compared to the SIR based algorithms. Furthermore, we show more generally, that LASSO succeeds in recovering the signed support of \boldsymbolβ0 if \boldsymbolX ∼ N(0, \boldsymbolΣ), and the covariance \boldsymbolΣ satisfies the irrepresentable condition. Our work extends existing results on the support recovery of LASSO for the linear model, to a more general class of SIMs.