Robustness May Be at Odds with Accuracy

2018/05/30 by Dimitris Tsipras, Tsipras, Dimitris, Shibani Santurkar +8 · 168 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #cs.CV #cs.LG #cs.NE #stat.ML

paper · pdf · doi:10.48550/arxiv.1805.12152

ICLR'19

openalex publication_date 2018/05/30 · arxiv created 2019/09/09 · arxiv updated 2019/09/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists in a fairly simple and natural setting. These findings also corroborate a similar phenomenon observed empirically in more complex settings. Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. These differences, in particular, seem to result in unexpected benefits: the representations learned by robust models tend to align better with salient data characteristics and human perception.

Citations

• Auto-Encoding Variational Bayes
• ImageNet classification with deep convolutional neural networks
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• A Dual Approach to Scalable Verification of Deep Networks
• Exploring the Landscape of Spatial Robustness
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Learning Multiple Layers of Features from Tiny Images

Cited by

• SafeMed-R1: Adversarial Reinforcement Learning for Generalizable and Robust Medical Reasoning in Vision-Language Models
• Stylized Synthetic Augmentation further improves Corruption Robustness
• Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability
• Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation
• TIE: A Training-Inversion-Exclusion Framework for Visually Interpretable and Uncertainty-Guided Out-of-Distribution Detection
• The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning
• Lost in Vagueness: Towards Context-Sensitive Standards for Robustness Assessment under the EU AI Act
• From Attribution to Action: Jointly ALIGNing Predictions and Explanations
• FrameShield: Adversarially Robust Video Anomaly Detection
• Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization
• H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
• Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models
• FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
• Nearly Space-Optimal Graph and Hypergraph Sparsification in Insertion-Only Data Streams
• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
• When Flatness Does (Not) Guarantee Adversarial Robustness
• Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training
• Feature Denoising for Improving Adversarial Robustness
• Improving Robustness using Generated Data
• Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning
• Adversarial Attacks Leverage Interference Between Features in Superposition
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
• Robust Regularization with Adversarial Labelling of Perturbed Samples
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• Architecture Selection via the Trade-off Between Accuracy and Robustness
• Machine-Learning Driven Load Shedding to Mitigate Instability Attacks in Power Grids
• Understanding Robustness in Teacher-Student Setting: A New Perspective
• Fidelity-Aware Data Composition for Robust Robot Generalization
• Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail
• Notes on Margin Training and Margin p-Values for Deep Neural Network Classifiers
• Achieving robustness in classification using optimal transport with hinge regularization
• SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
• Adversarial Examples Are Not Bugs, They Are Superposition
• What's the relationship between CNNs and communication systems?
• The Search for Sparse, Robust Neural Networks
• Rademacher Complexity for Adversarially Robust Generalization
• Natural Perturbed Training for General Robustness of Neural Network Classifiers
• Nearest Neighbor Projection Removal Adversarial Training
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• RobQFL: Robust Quantum Federated Learning in Adversarial Environment
• Defective Convolutional Networks
• A Brain-Inspired Gating Mechanism Unlocks Robust Computation in Spiking Neural Networks
• Motivating the Rules of the Game for Adversarial Example Research
• Sorting out Lipschitz function approximation
• A general framework for defining and optimizing robustness
• Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
• Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data
• Self-training Avoids Using Spurious Features Under Domain Shift
• Does simple trump complex? Comparing strategies for adversarial robustness in DNNs
• Adversarial Attacks on Monocular Depth Estimation
• Informative Post-Hoc Explanations Only Exist for Simple Functions
• Robust Convolution Neural ODEs via Contractivity-promoting regularization
• Adversarial Training for EM Classification Networks
• Robusta: Robust AutoML for Feature Selection via Reinforcement Learning
• An Experimental Study of Semantic Continuity for Deep Learning Models
• Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
• DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models
• An Integrated Approach to Produce Robust Models with High Efficiency
• Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networks
• Pulling Back the Curtain on Deep Networks
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Adversarial Robustness for Unsupervised Domain Adaptation
• Neural Architecture Dilation for Adversarial Robustness
• On the effectiveness of adversarial training against common corruptions
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
• Adversarial Training Improves Generalization Under Distribution Shifts in Bioacoustics
• Rethinking Natural Adversarial Examples for Classification Models
• On the Interaction of Compressibility and Adversarial Robustness
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification
• Enhancing Adversarial Defense by k-Winners-Take-All
• One Man's Trash is Another Man's Treasure: Resisting Adversarial Examples by Adversarial Examples
• Asymptotic Behavior of Adversarial Training in Binary Classification
• Emergence of Quantised Representations Isolated to Anisotropic Functions
• DeepBlur: A Simple and Effective Method for Natural Image Obfuscation
• β-Variational Classifiers Under Attack
• RobustBench: a standardized adversarial robustness benchmark
• LyAm: Robust Non-Convex Optimization for Stable Learning in Noisy Environments
• Adversarial Robustness Guarantees for Classification with Gaussian Processes
• PRM-Free Security Alignment of Large Models via Red Teaming and Adversarial Training
• On the Efficiency of Training Robust Decision Trees
• Explainable AI in Genomics: Transcription Factor Binding Site Prediction with Mixture of Experts
• Adversarial Examples Improve Image Recognition
• On the transferability of adversarial examples between convex and 01 loss models
• A Method for Computing Class-wise Universal Adversarial Perturbations
• Certifiably Robust Interpretation in Deep Learning
• How Does Mixup Help With Robustness and Generalization?
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• Mind the Detail: Uncovering Clinically Relevant Image Details in Accelerated MRI with Semantically Diverse Reconstructions
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency
• Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation
• Masked Autoencoders that Feel the Heart: Unveiling Simplicity Bias for ECG Analyses
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning
• DUMB and DUMBer: Is Adversarial Training Worth It in the Real World?
• Stretching Beyond the Obvious: A Gradient-Free Framework to Unveil the Hidden Landscape of Visual Invariance
• Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs
• Analytical Moment Regularizer for Gaussian Robust Networks
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Adversarial Robustness through Local Linearization
• CAAD 2018: Generating Transferable Adversarial Examples
• Active Adversarial Noise Suppression for Image Forgery Localization
• GrokAlign: Geometric Characterisation and Acceleration of Grokking
• On the Natural Robustness of Vision-Language Models Against Visual Perception Attacks in Autonomous Driving
• Bridging the Performance Gap between FGSM and PGD Adversarial Training
• Exploring Visual Prompting: Robustness Inheritance and Beyond
• An Empirical Analysis of the Impact of Data Augmentation on Knowledge Distillation
• Towards Understanding Adversarial Examples Systematically: Exploring Data Size, Task and Model Factors
• Intriguing properties of adversarial training at scale
• Towards Better Generalization via Distributional Input Projection Network
• X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP
• When Maximum Entropy Misleads Policy Optimization
• Identifying and Understanding Cross-Class Features in Adversarial Training
• Fundamental Tradeoffs in Distributionally Adversarial Training
• Adversarial Attack and Defense on Point Sets
• Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search
• Contrastive Learning with Adversarial Examples
• Towards Deep Learning Models Resistant to Large Perturbations
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Towards the Memorization Effect of Neural Networks in Adversarial Training
• Adversarial Examples in Modern Machine Learning: A Review
• Bridging the Gap Between Adversarial Robustness and Optimization Bias
• RobFR: Benchmarking Adversarial Robustness on Face Recognition
• Robustness in Both Domains: CLIP Needs a Robust Text Encoder
• Sampling Preferences Yields Simple Trustworthiness Scores
• Robust or Private? Adversarial Training Makes Models More Vulnerable to Privacy Attacks
• On the Scaling of Robustness and Effectiveness in Dense Retrieval
• Network Inversion for Uncertainty-Aware Out-of-Distribution Detection
• DISCO Verification: Division of Input Space into COnvex polytopes for neural network verification
• Balancing Robustness and Sensitivity using Feature Contrastive Learning
• TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularization
• On Robustness of Neural Semantic Parsers
• LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders
• Data Quality Matters For Adversarial Training: An Empirical Study
• Improving the Certified Robustness of Neural Networks via Consistency Regularization
• A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning
• COPYCAT: Practical Adversarial Attacks on Visualization-Based Malware Detection
• Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating
• Adversarially Pretrained Transformers may be Universally Robust In-Context Learners
• Adversarially Robust Low Dimensional Representations
• DynaNoise: Dynamic Probabilistic Noise Injection for Defending Against Membership Inference Attacks
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• Data-Agnostic Augmentations for Unknown Variations: Out-of-Distribution Generalisation in MRI Segmentation
• RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo
• ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training
• Adversarial Robustness May Be at Odds With Simplicity
• Improved Adversarial Training via Learned Optimizer
• Risk Analysis and Design Against Adversarial Actions
• Random Shadows and Highlights: A new data augmentation method for extreme lighting conditions
• Posterior Differential Regularization with f-divergence for Improving Model Robustness
• There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
• Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical World
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• DAmageNet: A Universal Adversarial Dataset
• A General Framework for Property-Driven Machine Learning
• On Connections between Regularizations for Improving DNN Robustness
• Supervised Learning Has a Geometric Blind Spot
• Design and Interpretation of Universal Adversarial Patches in Face Detection
• On the Benefits of Models with Perceptually-Aligned Gradients
• Estimating Principal Components under Adversarial Perturbations
• Purifying Adversarial Perturbation with Adversarially Trained Auto-encoders
• Robust Context-Aware Detection of Malicious Instructions in Text
• Defending Against Frequency-Based Attacks with Diffusion Models
• Adversarial Examples in Environment Perception for Automated Driving (Review)
• Empirical Upper Bound, Error Diagnosis and Invariance Analysis of Modern Object Detectors
• Trustworthy AI Must Account for Interactions
• Secure Diagnostics: Adversarial Robustness Meets Clinical Interpretability

Related

• Adversarially Robust Generalization Requires More Data
• What Do Adversarially Robust Models Look At?
• Adversarial Robustness May Be at Odds With Simplicity
• Intriguing Properties of Robust Classification
• The Pros and Cons of Adversarial Robustness