Adversarial Robustness via Label-Smoothing

2019/06/27 by Goibert, Morgane, Dohmatob, Elvis
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1906.11567

Abstract

We study Label-Smoothing as a means for improving adversarial robustness of supervised deep-learning models. After establishing a thorough and unified framework, we propose several variations to this general method: adversarial, Boltzmann and second-best Label-Smoothing methods, and we explain how to construct your own one. On various datasets (MNIST, CIFAR10, SVHN) and models (linear models, MLPs, LeNet, ResNet), we show that Label-Smoothing in general improves adversarial robustness against a variety of attacks (FGSM, BIM, DeepFool, Carlini-Wagner) by better taking account of the dataset geometry. The proposed Label-Smoothing methods have two main advantages: they can be implemented as a modified cross-entropy loss, thus do not require any modifications of the network architecture nor do they lead to increased training times, and they improve both standard and adversarial accuracy.

Related

• Label Smoothing and Adversarial Robustness
• Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement
• In and Out-of-Domain Text Adversarial Robustness via Label Smoothing