2019/07/26 by Kaiwen Wu, Wu, Kaiwen, Yaoliang Yu +1
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Physical Unclonable Functions (PUFs) and Hardware Security
paper · pdf · doi:10.48550/arxiv.1907.11780
openalex publication_date 2019/07/26 · openalex created_date 2019/08/13 · openalex updated_date 2026/07/28
Deep models, while being extremely versatile and accurate, are vulnerable to adversarial attacks: slight perturbations that are imperceptible to humans can completely flip the prediction of deep models. Many attack and defense mechanisms have been proposed, although a satisfying solution still largely remains elusive. In this work, we give strong evidence that during training, deep models maximize the minimum margin in order to achieve high accuracy, but at the same time decrease the average margin hence hurting robustness. Our empirical results highlight an intrinsic trade-off between accuracy and robustness for current deep model training. To further address this issue, we propose a new regularizer to explicitly promote average margin, and we verify through extensive experiments that it does lead to better robustness. Our regularized objective remains Fisher-consistent, hence asymptotically can still recover the Bayes optimal classifier.