Robust Principles: Architectural Design Principles for Adversarially Robust CNNs

2023/08/30 by Peng, ShengYun, Xu, Weilin, Cornelius, Cory +6 · 2 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2308.16258

Abstract

Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs. To accomplish our goal, we synthesize a suite of three generalizable robust architectural design principles: (a) optimal range for depth and width configurations, (b) preferring convolutional over patchify stem stage, and (c) robust residual block design through adopting squeeze and excitation blocks and non-parametric smooth activation functions. Through extensive experiments across a wide spectrum of dataset scales, adversarial training methods, model parameters, and network design spaces, our principles consistently and markedly improve AutoAttack accuracy: 1-3 percentage points (pp) on CIFAR-10 and CIFAR-100, and 4-9 pp on ImageNet. The code is publicly available at https://github.com/poloclub/robust-principles.

Cited by

• Dynamic Parameter Optimization for Highly Transferable Transformation-Based Attacks
• Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler

Related

• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
• Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspective
• Exploring the Relationship between Architecture and Adversarially Robust Generalization
• RobArch: Designing Robust Architectures against Adversarial Attacks
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks