Safe Screening for Logistic Regression with ℓ₀-ℓ₂ Regularization

2022/02/01 by Anna Deza, Deza, Anna, Alper Atamtürk +1 · 2 citations
Decision Sciences · Engineering · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Multi-Criteria Decision Making #Numerical methods in inverse problems #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.2202.00467

openalex publication_date 2022/02/01 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28

Abstract

In logistic regression, it is often desirable to utilize regularization to promote sparse solutions, particularly for problems with a large number of features compared to available labels. In this paper, we present screening rules that safely remove features from logistic regression with ℓ₀-ℓ₂ regularization before solving the problem. The proposed safe screening rules are based on lower bounds from the Fenchel dual of strong conic relaxations of the logistic regression problem. Numerical experiments with real and synthetic data suggest that a high percentage of the features can be effectively and safely removed apriori, leading to substantial speed-up in the computations.

Cited by

• A Generic Branch-and-Bound Algorithm for ℓ₀-Penalized Problems with Supplementary Material
• Safe screening rules for portfolio optimization with linear and cardinality constraints
• Screening Cut Generation for Sparse Ridge Regression

Related

• A Safe Screening Rule for Sparse Logistic Regression
• Safe Screening Rules for ℓ₀-Regression
• On Regularized Sparse Logistic Regression
• GAP Safe screening rules for sparse multi-task and multi-class models
• Gap Safe screening rules for sparsity enforcing penalties