Adversarial Machine Learning at Scale

2016/11/04 by Alexey Kurakin, Ian Goodfellow, Kurakin, Alexey +3 · 65 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1611.01236

openalex publication_date 2016/11/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Adversarial examples are malicious inputs designed to fool machine learning models. They often transfer from one model to another, allowing attackers to mount black box attacks without knowledge of the target model's parameters. Adversarial training is the process of explicitly training a model on adversarial examples, in order to make it more robust to attack or to reduce its test error on clean inputs. So far, adversarial training has primarily been applied to small problems. In this research, we apply adversarial training to ImageNet. Our contributions include: (1) recommendations for how to succesfully scale adversarial training to large models and datasets, (2) the observation that adversarial training confers robustness to single-step attack methods, (3) the finding that multi-step attack methods are somewhat less transferable than single-step attack methods, so single-step attacks are the best for mounting black-box attacks, and (4) resolution of a "label leaking" effect that causes adversarially trained models to perform better on adversarial examples than on clean examples, because the adversarial example construction process uses the true label and the model can learn to exploit regularities in the construction process.

Cited by

• I2VShield: An Efficient Proactive Defense Framework against DiT-based Image-to-Video Models
• Quickest Detection with Rough Path Signatures
• A New Kind of Adversarial Example: Measuring the Human-Model Gap, and Its Relationship to OOD Detection
• Defending against adversarial attacks using mixture of experts
• Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
• Generating Risky Samples with Conformity Constraints via Diffusion Models
• Adversarial Robustness in Zero-Shot Learning:An Empirical Study on Class and Concept-Level Vulnerabilities
• Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach
• Empirical evaluation of the Frank-Wolfe methods for constructing white-box adversarial attacks
• Creating Blank Canvas Against AI-enabled Image Forgery
• Neural surrogates for designing gravitational wave detectors
• Layer-wise Noise Guided Selective Wavelet Reconstruction for Robust Medical Image Segmentation
• On the Trade-Off Between Transparency and Security in Adversarial Machine Learning
• Class-feature Watermark: A Resilient Black-box Watermark Against Model Extraction Attacks
• Adapt under Attack and Domain Shift: Unified Adversarial Meta-Learning and Domain Adaptation for Robust Automatic Modulation Classification
• Dynamic Diameter in High-Dimensions against Adaptive Adversary and Beyond
• C-LEAD: Contrastive Learning for Enhanced Adversarial Defense
• Trans-defense: Transformer-based Denoiser for Adversarial Defense with Spatial-Frequency Domain Representation
• Bilevel Models for Adversarial Learning and A Case Study
• A Retraction-free Method for Nonsmooth Minimax Optimization over a Compact Manifold
• On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations
• FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
• A New Type of Adversarial Examples
• When Flatness Does (Not) Guarantee Adversarial Robustness
• NAPPure: Adversarial Purification for Robust Image Classification under Non-Additive Perturbations
• Towards Quantum Enhanced Adversarial Robustness with Rydberg Reservoir Learning
• Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning
• DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis
• Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail
• VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models
• Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
• DiffHash: Text-Guided Targeted Attack via Diffusion Models against Deep Hashing Image Retrieval
• Sy-FAR: Symmetry-based Fair Adversarial Robustness
• DARD: Dice Adversarial Robustness Distillation against Adversarial Attacks
• Probabilistic Robustness Analysis in High Dimensional Space: Application to Semantic Segmentation Network
• NAT: Learning to Attack Neurons for Enhanced Adversarial Transferability
• Ensemble Distribution Distillation for Self-Supervised Human Activity Recognition
• I Stolenly Swear That I Am Up to (No) Good: Design and Evaluation of Model Stealing Attacks
• Get Global Guarantees: On the Probabilistic Nature of Perturbation Robustness
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
• Extremal learning: extremizing the output of a neural network in regression problems
• Feature Losses for Adversarial Robustness
• A Survey of Black-Box Adversarial Attacks on Computer Vision Models
• ComplicitSplat: Downstream Models are Vulnerable to Blackbox Attacks by 3D Gaussian Splat Camouflages
• Mitigating Adversarial Effects Through Randomization
• Semantically Guided Adversarial Testing of Vision Models Using Language Models
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Input Hessian Regularization of Neural Networks
• Towards Effective Prompt Stealing Attack against Text-to-Image Diffusion Models
• ETA: Energy-based Test-time Adaptation for Depth Completion
• Non-Determinism in Neural Networks for Adversarial Robustness
• Superior resilience to poisoning and amenability to unlearning in quantum machine learning
• Disrupting Semantic and Abstract Features for Better Adversarial Transferability
• Max-Mahalanobis Linear Discriminant Analysis Networks
• Federated AI lets a team imagine together: Federated Learning of GANs
• Enhancing IoT Intrusion Detection Systems through Adversarial Training
• GraCIAS: Grassmannian of Corrupted Images for Adversarial Security
• Adversarial Attacks and Defenses: An Interpretation Perspective
• Robustness Certificates Against Adversarial Examples for ReLU Networks
• Spatial Frequency Modulation for Semantic Segmentation
• Wasserstein Introspective Neural Networks
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Theory-based residual neural networks: A synergy of discrete choice models and deep neural networks
• TRIX- Trading Adversarial Fairness via Mixed Adversarial Training

Related

• Adversarial Examples Make Strong Poisons
• Ensemble Adversarial Training: Attacks and Defenses
• Adversarial Examples in Modern Machine Learning: A Review
• One Man's Trash is Another Man's Treasure: Resisting Adversarial Examples by Adversarial Examples
• DUMB and DUMBer: Is Adversarial Training Worth It in the Real World?