2018/12/18 by Kian Kenyon-Dean, Andre Cianflone, Kenyon-Dean, Kian +10 · 5 citations
Computer Science · Mathematics · #62H30 #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Categorical variable #Cluster analysis #Computer science #Convolutional neural network #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Pattern recognition (psychology) #Representation (politics) #Similarity (geometry) #Topic Modeling #cs.AI #cs.LG #msc:62H30 #stat.ML
paper · pdf · doi:10.48550/arxiv.1812.07627
published in arXiv (Cornell University) (Cornell University) · AAAI 2019 Workshop on Network Interpretability for Deep Learning (9 pages)
arxiv created 2018/12/18 · openalex publication_date 2018/12/18 · arxiv updated 2018/12/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The standard loss function used to train neural network classifiers,\ncategorical cross-entropy (CCE), seeks to maximize accuracy on the training\ndata; building useful representations is not a necessary byproduct of this\nobjective. In this work, we propose clustering-oriented representation learning\n(COREL) as an alternative to CCE in the context of a generalized\nattractive-repulsive loss framework. COREL has the consequence of building\nlatent representations that collectively exhibit the quality of natural\nclustering within the latent space of the final hidden layer, according to a\npredefined similarity function. Despite being simple to implement, COREL\nvariants outperform or perform equivalently to CCE in a variety of scenarios,\nincluding image and news article classification using both feed-forward and\nconvolutional neural networks. Analysis of the latent spaces created with\ndifferent similarity functions facilitates insights on the different use cases\nCOREL variants can satisfy, where the Cosine-COREL variant makes a consistently\nclusterable latent space, while Gaussian-COREL consistently obtains better\nclassification accuracy than CCE.\n