Adversarial Robustness is at Odds with Lazy Training

2022/06/18 by Wang, Yunjuan, Ullah, Enayat, Mianjy, Poorya +1
#Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2207.00411

Abstract

Recent works show that adversarial examples exist for random neural networks [Daniely and Schacham, 2020] and that these examples can be found using a single step of gradient ascent [Bubeck et al., 2021]. In this work, we extend this line of work to "lazy training" of neural networks -- a dominant model in deep learning theory in which neural networks are provably efficiently learnable. We show that over-parametrized neural networks that are guaranteed to generalize well and enjoy strong computational guarantees remain vulnerable to attacks generated using a single step of gradient ascent.

Related

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Scaling up the randomized gradient-free adversarial attack reveals\n overestimation of robustness using established attacks
• Can Implicit Bias Imply Adversarial Robustness?
• Intriguing Properties of Adversarial Examples
• On the Properties of Adversarially-Trained CNNs