Extreme Value Preserving Networks

2020/11/17 by Mingjie Sun, Jianguo Li, Sun, Mingjie +3
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Bacillus and Francisella bacterial research #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2011.08367

arxiv created 2020/11/17 · openalex publication_date 2020/11/17 · arxiv updated 2020/11/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent evidence shows that convolutional neural networks (CNNs) are biased towards textures so that CNNs are non-robust to adversarial perturbations over textures, while traditional robust visual features like SIFT (scale-invariant feature transforms) are designed to be robust across a substantial range of affine distortion, addition of noise, etc with the mimic of human perception nature. This paper aims to leverage good properties of SIFT to renovate CNN architectures towards better accuracy and robustness. We borrow the scale-space extreme value idea from SIFT, and propose extreme value preserving networks (EVPNets). Experiments demonstrate that EVPNets can achieve similar or better accuracy than conventional CNNs, while achieving much better robustness on a set of adversarial attacks (FGSM,PGD,etc) even without adversarial training.

Citations

• ImageNet classification with deep convolutional neural networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Certified Adversarial Robustness via Randomized Smoothing
• Theoretically Principled Trade-off between Robustness and Accuracy
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models

Related

• The shape and simplicity biases of adversarially robust ImageNet-trained CNNs
• Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs
• Weight Map Layer for Noise and Adversarial Attack Robustness
• On the Adversarial Robustness of Vision Transformers
• Reveal of Vision Transformers Robustness against Adversarial Attacks