2022/05/05 by Sarat Moka, Moka, Sarat, Benoît Liquet +5 · 2 citations
Computer Science · Decision Sciences · Mathematics · #Algorithm #Computation (stat.CO) #Computer science #FOS: Computer and information sciences #Feature selection #Machine Learning and Data Classification #Machine learning #Mathematical optimization #Mathematics #Metaheuristic Optimization Algorithms Research #Methodology (stat.ME) #Multi-Criteria Decision Making #Optimization problem #Regression #Selection (genetic algorithm) #Set (abstract data type) #Statistics
paper · pdf · doi:10.48550/arxiv.2205.02617
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2022/05/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
The problem of best subset selection in linear regression is considered with the aim to find a fixed size subset of features that best fits the response. This is particularly challenging when the total available number of features is very large compared to the number of data samples. Existing optimal methods for solving this problem tend to be slow while fast methods tend to have low accuracy. Ideally, new methods perform best subset selection faster than existing optimal methods but with comparable accuracy, or, being more accurate than methods of comparable computational speed. Here, we propose a novel continuous optimization method that identifies a subset solution path, a small set of models of varying size, that consists of candidates for the single best subset of features, that is optimal in a specific sense in linear regression. Our method turns out to be fast, making the best subset selection possible when the number of features is well in excess of thousands. Because of the outstanding overall performance, framing the best subset selection challenge as a continuous optimization problem opens new research directions for feature extraction for a large variety of regression models.