2023/06/07 by Tomojit Ghosh, Ghosh, Tomojit, Michael Kirby +2
Biochemistry, Genetics and Molecular Biology · Engineering · Medicine · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Optical Imaging and Spectroscopy Techniques #Sparse and Compressive Sensing Techniques #Spectroscopy Techniques in Biomedical and Chemical Research
paper · pdf · doi:10.48550/arxiv.2306.04824
openalex publication_date 2023/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a novel feature selection technique, Sparse Linear Centroid-Encoder (SLCE). The algorithm uses a linear transformation to reconstruct a point as its class centroid and, at the same time, uses the ℓ1-norm penalty to filter out unnecessary features from the input data. The original formulation of the optimization problem is nonconvex, but we propose a two-step approach, where each step is convex. In the first step, we solve the linear Centroid-Encoder, a convex optimization problem over a matrix A. In the second step, we only search for a sparse solution over a diagonal matrix B while keeping A fixed. Unlike other linear methods, e.g., Sparse Support Vector Machines and Lasso, Sparse Linear Centroid-Encoder uses a single model for multi-class data. We present an in-depth empirical analysis of the proposed model and show that it promotes sparsity on various data sets, including high-dimensional biological data. Our experimental results show that SLCE has a performance advantage over some state-of-the-art neural network-based feature selection techniques.