Feature Denoising for Improving Adversarial Robustness

2018/12/09 by Cihang Xie, Xie, Cihang, Yuxin Wu +7 · 16 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Bacillus and Francisella bacterial research #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.1812.03411

openalex publication_date 2018/12/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Adversarial attacks to image classification systems present challenges to convolutional networks and opportunities for understanding them. This study suggests that adversarial perturbations on images lead to noise in the features constructed by these networks. Motivated by this observation, we develop new network architectures that increase adversarial robustness by performing feature denoising. Specifically, our networks contain blocks that denoise the features using non-local means or other filters; the entire networks are trained end-to-end. When combined with adversarial training, our feature denoising networks substantially improve the state-of-the-art in adversarial robustness in both white-box and black-box attack settings. On ImageNet, under 10-iteration PGD white-box attacks where prior art has 27.9% accuracy, our method achieves 55.7%; even under extreme 2000-iteration PGD white-box attacks, our method secures 42.6% accuracy. Our method was ranked first in Competition on Adversarial Attacks and Defenses (CAAD) 2018 --- it achieved 50.6% classification accuracy on a secret, ImageNet-like test dataset against 48 unknown attackers, surpassing the runner-up approach by ~10%. Code is available at https://github.com/facebookresearch/ImageNet-Adversarial-Training.

Citations

• ImageNet Large Scale Visual Recognition Challenge
• Backpropagation Applied to Handwritten Zip Code Recognition
• ImageNet classification with deep convolutional neural networks
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Robustness May Be at Odds with Accuracy

Cited by

• Sample-wise Adaptive Weighting for Transfer Consistency in Adversarial Distillation
• AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing
• Calibrated Adversarial Sampling: Multi-Armed Bandit-Guided Generalization Against Unforeseen Attacks
• DeepDefense: Layer-Wise Gradient-Feature Alignment for Building Robust Neural Networks
• Improving Sustainability of Adversarial Examples in Class-Incremental Learning
• Deep learning models are vulnerable, but adversarial examples are even more vulnerable
• Beyond Deceptive Flatness: Dual-Order Solution for Strengthening Adversarial Transferability
• Trustworthy Quantum Machine Learning: A Roadmap for Reliability, Robustness, and Security in the NISQ Era
• ANCHOR: Integrating Adversarial Training with Hard-mined Supervised Contrastive Learning for Robust Representation Learning
• Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensembles
• Towards Strong Certified Defense with Universal Asymmetric Randomization
• Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
• RanAT4BIE: Random Adversarial Training for Biomedical Information Extraction
• An Investigation of Visual Foundation Models Robustness
• Jamming Identification with Differential Transformer for Low-Altitude Wireless Networks
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective

Related

• Feature Losses for Adversarial Robustness
• Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack
• Towards Adversarial Purification using Denoising AutoEncoders
• Divide, Denoise, and Defend against Adversarial Attacks
• Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack