Sparse Discriminant Analysis

2011/11/01 by Line Clemmensen, Line Katrine Harder Clemmensen, Trevor Hastie +3 · 554 citations
Biochemistry, Genetics and Molecular Biology · Chemistry · Computer Science · Mathematics · #Artificial intelligence #Computer science #Discriminant #Face and Expression Recognition #Gene expression and cancer classification #Linear discriminant analysis #Mathematics #Multiple discriminant analysis #Optimal discriminant analysis #Pattern recognition (psychology) #Spectroscopy and Chemometric Analyses #Statistics

paper · doi:10.1198/tech.2011.08118

published in Technometrics 53(4), 406-413 (Taylor & Francis)

openalex publication_date 2011/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

We consider the problem of performing interpretable classification in the high-dimensional setting, in which the number of features is very large and the number of observations is limited. This setting has been studied extensively in the chemometrics literature, and more recently has become commonplace in biological and medical applications. In this setting, a traditional approach involves performing feature selection before classification. We propose sparse discriminant analysis, a method for performing linear discriminant analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously. Sparse discriminant analysis is based on the optimal scoring interpretation of linear discriminant analysis, and can be extended to perform sparse discrimination via mixtures of Gaussians if boundaries between classes are nonlinear or if subgroups are present within each class. Our proposal also provides low-dimensional views of the discriminative directions.

Citations

• Least angle regression

Cited by

• An Efficient Approach to Sparse Linear Discriminant Analysis
• Variational Discriminant Analysis with Variable Selection
• Group-sparse SVD Models and Their Applications in Biological Data
• Robust Variable and Interaction Selection for Logistic Regression and Multiple Index Models
• On missing label patterns in semi-supervised learning
• Robust Sparse Linear Discriminant Analysis
• Unobserved classes and extra variables in high-dimensional discriminant analysis
• Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
• Whole-brain Prediction Analysis with GraphNet
• Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images
• Multiclass Sparse Discriminant Analysis
• Graph embeddings of dynamic functional connectivity reveal discriminative patterns of task engagement in HCP data
• Covariate-Adjusted Tensor Classification in High-Dimensions
• Multi-Spectral Imaging via Computed Tomography (MUSIC) - Comparing Unsupervised Spectral Segmentations for Material Differentiation
• Sparse Fisher's Linear Discriminant Analysis for Partially Labeled Data
• Robust Locality-Aware Regression for Labeled Data Classification
• Alternating direction method of multipliers for penalized zero-variance discriminant analysis
• Semiparametric Sparse Discriminant Analysis
• Half-CNN: A General Framework for Whole-Image Regression
• Supervised Discriminative Sparse PCA for Com-Characteristic Gene Selection and Tumor Classification on Multiview Biological Data
• Deep Learning and Machine Vision for Food Processing: A Survey
• Sparse Fisher's discriminant analysis with thresholded linear constraints
• Asymptotic optimality of sparse linear discriminant analysis with arbitrary number of classes
• Statistical Methods and Workflow for Analyzing Human Metabolomics Data
• An intelligent non-optimality self-recovery method based on reinforcement learning with small data in big data era
• Continuous decoding of movement intention of upper limb self-initiated analytic movements from pre-movement EEG correlates. [europepmc]
• Optimal Feature Selection in High-Dimensional Discriminant Analysis. [europepmc]
• Variational cross-validation of slow dynamical modes in molecular kinetics. [europepmc]
• Number of genes controlling a quantitative trait in a hybrid zone of the aposematic frog Ranitomeya imitator. [europepmc]
• Detecting intention to walk in stroke patients from pre-movement EEG correlates. [europepmc]
• Control of an Ambulatory Exoskeleton with a Brain-Machine Interface for Spinal Cord Injury Gait Rehabilitation. [europepmc]
• Decoding Time-Varying Functional Connectivity Networks via Linear Graph Embedding Methods. [europepmc]
• MultiLink Analysis: Brain Network Comparison via Sparse Connectivity Analysis. [europepmc]
• Brain connectivity alterations in early psychosis: from clinical to neuroimaging staging. [europepmc]
• Abnormal Phase Coupling in Parkinson's Disease and Normalization Effects of Subthreshold Vestibular Stimulation. [europepmc]
• Statistical Workflow for Feature Selection in Human Metabolomics Data. [europepmc]
• A Generic Multivariate Framework for the Integration of Microbiome Longitudinal Studies With Other Data Types. [europepmc]
• Multiband entropy-based feature-extraction method for automatic identification of epileptic focus based on high-frequency components in interictal iEEG. [europepmc]
• The Ancestral Caenorhabditis elegans Cuticle Suppresses rol-1 . [europepmc]
• Cigarette Smoking during Pregnancy: Effects on Antioxidant Enzymes, Metallothionein and Trace Elements in Mother-Newborn Pairs. [europepmc]
• Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review. [europepmc]
• Classification Accuracy of Hepatitis C Virus Infection Outcome: Data Mining Approach. [europepmc]
• Statistical and Machine Learning Link Selection Methods for Brain Functional Networks: Review and Comparison. [europepmc]
• Benchmarking Eliminative Radiomic Feature Selection for Head and Neck Lymph Node Classification. [europepmc]
• Prediction of disease progression indicators in prostate cancer patients receiving HDR-brachytherapy using Raman spectroscopy and semi-supervised learning: a pilot study. [europepmc]
• Systems approach for congruence and selection of cancer models towards precision medicine. [europepmc]
• Stacking Model-Based Classifiers for Dealing With Multiple Sets of Noisy Labels. [europepmc]
• Sparse vertex discriminant analysis: Variable selection for biomedical classification applications. [europepmc]
• Machine Learning-Based Classification of Cervical Lymph Nodes in HNSCC: A Radiomics Approach with Feature Selection Optimization. [europepmc]
• MTHFR 677C/T gene polymorphism and dietary habits: effects on trace element levels, amino acids, and biochemical parameters. [europepmc]

Related

• An Efficient Approach to Sparse Linear Discriminant Analysis
• Optimal Feature Selection in High-Dimensional Discriminant Analysis
• Semiparametric Sparse Discriminant Analysis
• Multiclass Sparse Discriminant Analysis
• A Direct Estimation Approach to Sparse Linear Discriminant Analysis