Sparse DNNs with Improved Adversarial Robustness

2018/10/23 by Yiwen Guo, Guo, Yiwen, Chao Zhang +5 · 5 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Cryptography and Security (cs.CR) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1810.09619

openalex publication_date 2018/10/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep neural networks (DNNs) are computationally/memory-intensive and vulnerable to adversarial attacks, making them prohibitive in some real-world applications. By converting dense models into sparse ones, pruning appears to be a promising solution to reducing the computation/memory cost. This paper studies classification models, especially DNN-based ones, to demonstrate that there exists intrinsic relationships between their sparsity and adversarial robustness. Our analyses reveal, both theoretically and empirically, that nonlinear DNN-based classifiers behave differently under l₂ attacks from some linear ones. We further demonstrate that an appropriately higher model sparsity implies better robustness of nonlinear DNNs, whereas over-sparsified models can be more difficult to resist adversarial examples.

Citations

• Spectrally-normalized margin bounds for neural networks
• Caffe: Convolutional Architecture for Fast Feature Embedding

Cited by

• Visual Token Compression Enhances Robustness of MLLMs
• Robust Experts: the Effect of Adversarial Training on CNNs with Sparse Mixture-of-Experts Layers
• Saddle Hierarchy in Dense Associative Memory
• Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation
• Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning

Related

• Combating Adversarial Attacks Using Sparse Representations
• Evaluating Model Robustness Using Adaptive Sparse L0 Regularization
• Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness
• Pruning in the Face of Adversaries
• The Search for Sparse, Robust Neural Networks