2021/01/23 by Blaž Škrlj, Sašo Džeroski, Škrlj, Blaž +5
Biochemistry, Genetics and Molecular Biology · Computer Science · #FOS: Computer and information sciences #Face and Expression Recognition #Gene expression and cancer classification #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2101.09577
openalex publication_date 2021/01/23 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Feature ranking has been widely adopted in machine learning applications such\nas high-throughput biology and social sciences. The approaches of the popular\nRelief family of algorithms assign importances to features by iteratively\naccounting for nearest relevant and irrelevant instances. Despite their high\nutility, these algorithms can be computationally expensive and not-well suited\nfor high-dimensional sparse input spaces. In contrast, recent embedding-based\nmethods learn compact, low-dimensional representations, potentially\nfacilitating down-stream learning capabilities of conventional learners. This\npaper explores how the Relief branch of algorithms can be adapted to benefit\nfrom (Riemannian) manifold-based embeddings of instance and target spaces,\nwhere a given embedding's dimensionality is intrinsic to the dimensionality of\nthe considered data set. The developed ReliefE algorithm is faster and can\nresult in better feature rankings, as shown by our evaluation on 20 real-life\ndata sets for multi-class and multi-label classification tasks. The utility of\nReliefE for high-dimensional data sets is ensured by its implementation that\nutilizes sparse matrix algebraic operations. Finally, the relation of ReliefE\nto other ranking algorithms is studied via the Fuzzy Jaccard Index.\n