An Estimator for the Sensitivity to Perturbations of Deep Neural Networks

2023/07/24 by Naman Maheshwari, Nicholas Malaya, Maheshwari, Naman +7
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Analysis (math.NA)

paper · pdf · doi:10.48550/arxiv.2307.12679

openalex publication_date 2023/07/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

For Deep Neural Networks (DNNs) to become useful in safety-critical applications, such as self-driving cars and disease diagnosis, they must be stable to perturbations in input and model parameters. Characterizing the sensitivity of a DNN to perturbations is necessary to determine minimal bit-width precision that may be used to safely represent the network. However, no general result exists that is capable of predicting the sensitivity of a given DNN to round-off error, noise, or other perturbations in input. This paper derives an estimator that can predict such quantities. The estimator is derived via inequalities and matrix norms, and the resulting quantity is roughly analogous to a condition number for the entire neural network. An approximation of the estimator is tested on two Convolutional Neural Networks, AlexNet and VGG-19, using the ImageNet dataset. For each of these networks, the tightness of the estimator is explored via random perturbations and adversarial attacks.

Related

• There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
• SAfER: Layer-Level Sensitivity Assessment for Efficient and Robust Neural Network Inference
• An Explainable Adversarial Robustness Metric for Deep Learning Neural Networks
• gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks
• Towards Robust Deep Neural Networks