MMA Training: Direct Input Space Margin Maximization through Adversarial\n Training

2018/12/06 by Gavin Weiguang Ding, Yash Sharma, Ding, Gavin Weiguang +5 · 18 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Advanced Malware Detection Techniques

paper · pdf · doi:10.48550/arxiv.1812.02637

Abstract

We study adversarial robustness of neural networks from a margin maximization\nperspective, where margins are defined as the distances from inputs to a\nclassifier's decision boundary. Our study shows that maximizing margins can be\nachieved by minimizing the adversarial loss on the decision boundary at the\n"shortest successful perturbation", demonstrating a close connection between\nadversarial losses and the margins. We propose Max-Margin Adversarial (MMA)\ntraining to directly maximize the margins to achieve adversarial robustness.\nInstead of adversarial training with a fixed \ε, MMA offers an\nimprovement by enabling adaptive selection of the "correct" \ε as the\nmargin individually for each datapoint. In addition, we rigorously analyze\nadversarial training with the perspective of margin maximization, and provide\nan alternative interpretation for adversarial training, maximizing either a\nlower or an upper bound of the margins. Our experiments empirically confirm our\ntheory and demonstrate MMA training's efficacy on the MNIST and CIFAR10\ndatasets w.r.t. \ℓ_\∞ and \ℓ₂ robustness. Code and models are\navailable at https://github.com/BorealisAI/mma_training.\n

Citations

• Improving the Adversarial Robustness and Interpretability of Deep Neural\n Networks by Regularizing their Input Gradients

Cited by

• Parameter Interpolation Adversarial Training for Robust Image Classification
• Recent Advances in Adversarial Training for Adversarial Robustness
• LAFEAT: Piercing Through Adversarial Defenses with Latent Features
• Tight Robustness Certificates and Wasserstein Distributional Attacks for Deep Neural Networks
• Bag of Tricks for Adversarial Training
• Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Adversarial Scheduling
• Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
• SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
• Robustifying Diffusion-Denoised Smoothing Against Covariate Shift
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
• Does simple trump complex? Comparing strategies for adversarial robustness in DNNs
• Enhanced countering adversarial attacks via input denoising and feature restoring
• Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Training
• Theoretical Analysis of Relative Errors in Gradient Computations for Adversarial Attacks with CE Loss
• Adaptive perturbation adversarial training: based on reinforcement learning
• RobustBench: a standardized adversarial robustness benchmark
• TRIX- Trading Adversarial Fairness via Mixed Adversarial Training
• Enhance CNN Robustness Against Noises for Classification of 12-Lead ECG with Variable Length

Related

• Towards the Desirable Decision Boundary by Moderate-Margin Adversarial Training
• Building Robust Ensembles via Margin Boosting
• Adversarial Training in Low-Label Regimes with Margin-Based Interpolation
• Bridged Adversarial Training
• Splitting the Difference on Adversarial Training