Towards Defending Multiple ℓ_p-norm Bounded Adversarial Perturbations via Gated Batch Normalization

2020/12/03 by Aishan Liu, Liu, Aishan, Shiyu Tang +10
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Adversarial system #Anomaly Detection Techniques and Applications #Artificial intelligence #Artificial neural network #Bounded function #Combinatorics #Computer Vision and Pattern Recognition (cs.CV) #Computer science #FOS: Computer and information sciences #MNIST database #Mathematical analysis #Mathematics #Norm (philosophy) #Normalization (sociology) #Perturbation (astronomy) #Physics

paper · pdf · doi:10.48550/arxiv.2012.01654

openalex publication_date 2020/12/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

There has been extensive evidence demonstrating that deep neural networks are vulnerable to adversarial examples, which motivates the development of defenses against adversarial attacks. Existing adversarial defenses typically improve model robustness against individual specific perturbation types (\eg, ℓ_∞-norm bounded adversarial examples). However, adversaries are likely to generate multiple types of perturbations in practice (\eg, ℓ₁, ℓ₂, and ℓ_∞ perturbations). Some recent methods improve model robustness against adversarial attacks in multiple ℓ_p balls, but their performance against each perturbation type is still far from satisfactory. In this paper, we observe that different ℓ_p bounded adversarial perturbations induce different statistical properties that can be separated and characterized by the statistics of Batch Normalization (BN). We thus propose Gated Batch Normalization (GBN) to adversarially train a perturbation-invariant predictor for defending multiple ℓ_p bounded adversarial perturbations. GBN consists of a multi-branch BN layer and a gated sub-network. Each BN branch in GBN is in charge of one perturbation type to ensure that the normalized output is aligned towards learning perturbation-invariant representation. Meanwhile, the gated sub-network is designed to separate inputs added with different perturbation types. We perform an extensive evaluation of our approach on commonly-used dataset including MNIST, CIFAR-10, and Tiny-ImageNet, and demonstrate that GBN outperforms previous defense proposals against multiple perturbation types (\ie, ℓ₁, ℓ₂, and ℓ_∞ perturbations) by large margins.

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Explaining and Harnessing Adversarial Examples
• Gradient-based learning applied to document recognition
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• ImageNet classification with deep convolutional neural networks
• Adversarial Examples in the Physical World
• On Detecting Adversarial Perturbations
• Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Testing Robustness Against Unforeseen Adversaries
• Training Robust Deep Neural Networks via Adversarial Noise Propagation
• Learning Multiple Layers of Features from Tiny Images

Related

• Towards an Adversarially Robust Normalization Approach
• Batch Normalization is a Cause of Adversarial Vulnerability
• Adaptive Batch Normalization Networks for Adversarial Robustness
• Beneficial Perturbations Network for Defending Adversarial Examples
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples