2019/03/22 by S. Jalil Kazemitabar, Arash Amini, Kazemitabar, S. Jalil +3
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Mathematics #Gene expression and cancer classification #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1903.09488
openalex publication_date 2019/03/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Leading methods for support recovery in high-dimensional regression, such as Lasso, have been well-studied and their limitations in the context of correlated design have been characterized with precise incoherence conditions. In this work, we present a similar treatment of selection consistency for marginal regression (MR), a computationally efficient family of methods with connections to decision trees. Selection based on marginal regression is also referred to as covariate screening or independence screening and is a popular approach in applied work, especially in ultra high-dimensional settings. We identify the underlying factors---which we denote as MR incoherence---affecting MR's support recovery performance. Our near complete characterization provides a much more nuanced and optimistic view of MR in comparison to previous works. To ground our results, we provide a broad taxonomy of results for leading feature selection methods, relating the behavior of Lasso, OMP, SIS, and MR. We also lay the foundation for interesting generalizations of our analysis, e.g., to non-linear feature selection methods and to more general regression frameworks such as a general additive models.