Robustifying ℓ_∞ Adversarial Training to the Union of Perturbation Models

2021/05/31 by Ameya D. Patil, Patil, Ameya D., Michael Tuttle +5
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Bacillus and Francisella bacterial research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2105.14710

openalex publication_date 2021/05/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Classical adversarial training (AT) frameworks are designed to achieve high adversarial accuracy against a single attack type, typically ℓ_∞ norm-bounded perturbations. Recent extensions in AT have focused on defending against the union of multiple perturbations but this benefit is obtained at the expense of a significant (up to 10×) increase in training complexity over single-attack ℓ_∞ AT. In this work, we expand the capabilities of widely popular single-attack ℓ_∞ AT frameworks to provide robustness to the union of (ℓ_∞, ℓ₂, ℓ₁) perturbations while preserving their training efficiency. Our technique, referred to as Shaped Noise Augmented Processing (SNAP), exploits a well-established byproduct of single-attack AT frameworks -- the reduction in the curvature of the decision boundary of networks. SNAP prepends a given deep net with a shaped noise augmentation layer whose distribution is learned along with network parameters using any standard single-attack AT. As a result, SNAP enhances adversarial accuracy of ResNet-18 on CIFAR-10 against the union of (ℓ_∞, ℓ₂, ℓ₁) perturbations by 14%-to-20% for four state-of-the-art (SOTA) single-attack ℓ_∞ AT frameworks, and, for the first time, establishes a benchmark for ResNet-50 and ResNet-101 on ImageNet.

Citations

• Certified Adversarial Robustness via Randomized Smoothing
• Adversarial Examples Are Not Bugs, They Are Features
• Exploring the Landscape of Spatial Robustness
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
• Model Compression with Adversarial Robustness: A Unified Optimization\n Framework
• On Adaptive Attacks to Adversarial Example Defenses
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples

Related

• \ℓ_\∞-Robustness and Beyond: Unleashing Efficient Adversarial\n Training
• Improved Adversarial Training Through Adaptive Instance-wise Loss Smoothing
• Towards Alternative Techniques for Improving Adversarial Robustness: Analysis of Adversarial Training at a Spectrum of Perturbations
• Parameter-Saving Adversarial Training: Reinforcing Multi-Perturbation Robustness via Hypernetworks
• Ensemble Adversarial Training: Attacks and Defenses