Robustness May Be at Odds with Accuracy

2018/05/30 by Dimitris Tsipras, Tsipras, Dimitris, Shibani Santurkar +7 · 63 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1805.12152

openalex publication_date 2018/05/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists in a fairly simple and natural setting. These findings also corroborate a similar phenomenon observed empirically in more complex settings. Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. These differences, in particular, seem to result in unexpected benefits: the representations learned by robust models tend to align better with salient data characteristics and human perception.

Citations

• Auto-Encoding Variational Bayes
• ImageNet classification with deep convolutional neural networks
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• A Dual Approach to Scalable Verification of Deep Networks
• Exploring the Landscape of Spatial Robustness
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Improving the Adversarial Robustness and Interpretability of Deep Neural\n Networks by Regularizing their Input Gradients
• Learning Multiple Layers of Features from Tiny Images

Cited by

• SafeMed-R1: Adversarial Reinforcement Learning for Generalizable and Robust Medical Reasoning in Vision-Language Models
• Stylized Synthetic Augmentation further improves Corruption Robustness
• Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability
• Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation
• TIE: A Training-Inversion-Exclusion Framework for Visually Interpretable and Uncertainty-Guided Out-of-Distribution Detection
• The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning
• Lost in Vagueness: Towards Context-Sensitive Standards for Robustness Assessment under the EU AI Act
• From Attribution to Action: Jointly ALIGNing Predictions and Explanations
• FrameShield: Adversarially Robust Video Anomaly Detection
• Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization
• H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
• Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models
• FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
• Nearly Space-Optimal Graph and Hypergraph Sparsification in Insertion-Only Data Streams
• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
• When Flatness Does (Not) Guarantee Adversarial Robustness
• Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training
• Feature Denoising for Improving Adversarial Robustness
• Improving Robustness using Generated Data
• Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning
• Adversarial Attacks Leverage Interference Between Features in Superposition
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
• Robust Regularization with Adversarial Labelling of Perturbed Samples
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• Architecture Selection via the Trade-off Between Accuracy and Robustness
• Machine-Learning Driven Load Shedding to Mitigate Instability Attacks in Power Grids
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• Fidelity-Aware Data Composition for Robust Robot Generalization
• Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail
• Notes on Margin Training and Margin p-Values for Deep Neural Network Classifiers
• Achieving robustness in classification using optimal transport with hinge regularization
• SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
• Adversarial Examples Are Not Bugs, They Are Superposition
• What's the relationship between CNNs and communication systems?
• The Search for Sparse, Robust Neural Networks
• Rademacher Complexity for Adversarially Robust Generalization
• Natural Perturbed Training for General Robustness of Neural Network\n Classifiers
• Nearest Neighbor Projection Removal Adversarial Training
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• RobQFL: Robust Quantum Federated Learning in Adversarial Environment
• Defective Convolutional Networks
• A Brain-Inspired Gating Mechanism Unlocks Robust Computation in Spiking Neural Networks
• Motivating the Rules of the Game for Adversarial Example Research
• Sorting out Lipschitz function approximation
• A general framework for defining and optimizing robustness
• Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
• Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data
• Self-training Avoids Using Spurious Features Under Domain Shift
• Does simple trump complex? Comparing strategies for adversarial robustness in DNNs
• Adversarial Attacks on Monocular Depth Estimation
• Informative Post-Hoc Explanations Only Exist for Simple Functions
• Robust Convolution Neural ODEs via Contractivity-promoting regularization
• Adversarial Training for EM Classification Networks
• Robusta: Robust AutoML for Feature Selection via Reinforcement Learning
• An Experimental Study of Semantic Continuity for Deep Learning Models
• Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
• DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models
• An Integrated Approach to Produce Robust Models with High Efficiency
• Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networks
• Pulling Back the Curtain on Deep Networks
• Adversarial Attacks and Defenses: An Interpretation Perspective

Related

• Adversarially Robust Generalization Requires More Data
• What Do Adversarially Robust Models Look At?
• Adversarial Robustness May Be at Odds With Simplicity
• Intriguing Properties of Robust Classification
• The Pros and Cons of Adversarial Robustness