Introducing Graph Smoothness Loss for Training Deep Learning\n Architectures

2019/05/01 by Myriam Bontonou, Carlos Lassance, Bontonou, Myriam +9
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1905.00301

openalex publication_date 2019/05/01 · openalex created_date 2022/07/23 · openalex updated_date 2026/07/28

Abstract

We introduce a novel loss function for training deep learning architectures\nto perform classification. It consists in minimizing the smoothness of label\nsignals on similarity graphs built at the output of the architecture.\nEquivalently, it can be seen as maximizing the distances between the network\nfunction images of training inputs from distinct classes. As such, only\ndistances between pairs of examples in distinct classes are taken into account\nin the process, and the training does not prevent inputs from the same class to\nbe mapped to distant locations in the output domain. We show that this loss\nleads to similar performance in classification as architectures trained using\nthe classical cross-entropy, while offering interesting degrees of freedom and\nproperties. We also demonstrate the interest of the proposed loss to increase\nrobustness of trained architectures to deviations of the inputs.\n

Related

• Leveraging Class Similarity to Improve Deep Neural Network Robustness
• Laplacian Networks: Bounding Indicator Function Smoothness for Neural Network Robustness
• Reduced Jeffries-Matusita distance: A Novel Loss Function to Improve Generalization Performance of Deep Classification Models
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Neural networks adapting to datasets: learning network size and topology