Boosting Adversarial Training with Hypersphere Embedding

2020/02/20 by Tianyu Pang, Xiao Yang, Pang, Tianyu +9 · 4 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2002.08619

openalex publication_date 2020/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models. In this work, we advocate incorporating the hypersphere embedding (HE) mechanism into the AT procedure by regularizing the features onto compact manifolds, which constitutes a lightweight yet effective module to blend in the strength of representation learning. Our extensive analyses reveal that AT and HE are well coupled to benefit the robustness of the adversarially trained models from several aspects. We validate the effectiveness and adaptability of HE by embedding it into the popular AT frameworks including PGD-AT, ALP, and TRADES, as well as the FreeAT and FastAT strategies. In the experiments, we evaluate our methods under a wide range of adversarial attacks on the CIFAR-10 and ImageNet datasets, which verifies that integrating HE can consistently enhance the model robustness for each AT framework with little extra computation.

Citations

• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Improving Adversarial Robustness via Promoting Ensemble Diversity
• On Evaluating Adversarial Robustness
• On the Convergence and Robustness of Adversarial Training
• Adversarial Examples Are a Natural Consequence of Test Error in Noise
• Practical Black-Box Attacks against Machine Learning
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Metric Learning for Adversarial Robustness
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• An Alternative Surrogate Loss for PGD-based Adversarial Testing
• Improving Machine Reading Comprehension via Adversarial Training
• Fast is better than free: Revisiting adversarial training
• Benchmarking Adversarial Robustness
• Overfitting in adversarially robust deep learning
• RayS: A Ray Searching Method for Hard-label Adversarial Attack
• Bag of Tricks for Adversarial Training
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Sharpness-Aware Geometric Defense for Robust Out-Of-Distribution Detection
• RCR-AF: Enhancing Model Generalization via Rademacher Complexity Reduction Activation Function
• Theoretical Analysis of Relative Errors in Gradient Computations for Adversarial Attacks with CE Loss
• Towards Class-wise Fair Adversarial Training via Anti-Bias Soft Label Distillation

Related

• Improving Adversarial Robustness with Hypersphere Embedding and Angular-based Regularizations
• Approximate Manifold Defense Against Multiple Adversarial Perturbations
• Manifold attack
• Improving Model Robustness with Latent Distribution Locally and Globally
• Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks