Random Subspace with Trees for Feature Selection Under Memory Constraints

2017/09/04 by Antonio Sutera, Célia Châtel, Sutera, Antonio +7
Computer Science · Mathematics · #Algorithms and Data Compression #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1709.01177

openalex publication_date 2017/09/04 · arxiv created 2017/09/06 · arxiv updated 2017/09/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Dealing with datasets of very high dimension is a major challenge in machine learning. In this paper, we consider the problem of feature selection in applications where the memory is not large enough to contain all features. In this setting, we propose a novel tree-based feature selection approach that builds a sequence of randomized trees on small subsamples of variables mixing both variables already identified as relevant by previous models and variables randomly selected among the other variables. As our main contribution, we provide an in-depth theoretical analysis of this method in infinite sample setting. In particular, we study its soundness with respect to common definitions of feature relevance and its convergence speed under various variable dependance scenarios. We also provide some preliminary empirical results highlighting the potential of the approach.

Citations

• Approximate False Positive Rate Control in Selection Frequency for Random Forest
• Wrappers for feature subset selection

Related

• Nonparametric Feature Selection by Random Forests and Deep Neural Networks
• On feature selection in double-imbalanced data settings: a Random Forest approach
• Feature Selection Methods for Cost-Constrained Classification in Random Forests
• Feature and Variable Selection in Classification
• Sequential Feature Classification in the Context of Redundancies