Generalized Fisher Score for Feature Selection

2012/02/14 by Quanquan Gu, Gu, Quanquan, Zhenhui Li +3 · 10 citations
Computer Science · #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1202.3725

openalex publication_date 2012/02/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.

Cited by

• Heatmap Pooling Network for Action Recognition from RGB Videos
• Rethinking Cross-Generator Image Forgery Detection through DINOv3
• Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions
• Training-Free Time Series Classification via In-Context Reasoning with LLM Agents
• LLM-FS-Agent: A Deliberative Role-based Large Language Model Architecture for Transparent Feature Selection
• KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning
• Modular Arithmetic: Language Models Solve Math Digit by Digit
• Physics-guided denoiser network for enhanced additive manufacturing data quality
• Where to Intervene: Action Selection in Deep Reinforcement Learning
• Predicting antidepressant responsiveness in major depressive disorder patients via electroencephalography gamma-band dynamic functional connectivity in response to salient auditory stimuli

Related

• Automated Model Selection for Generalized Linear Models
• Selection by Partitioning the Solution Paths
• Gradient Boosted Feature Selection
• Supervised feature selection with orthogonal regression and feature weighting
• Supervised Infinite Feature Selection