There is Limited Correlation between Coverage and Robustness for Deep Neural Networks

2019/11/14 by Yizhen Dong, Dong, Yizhen, Peixin Zhang +17
Computer Science · Mathematics · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Software Engineering (cs.SE) #Software Testing and Debugging Techniques #cs.LG #cs.SE #stat.ML

paper · pdf · doi:10.48550/arxiv.1911.05904

arxiv created 2019/11/14 · openalex publication_date 2019/11/14 · arxiv updated 2019/11/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep neural networks (DNN) are increasingly applied in safety-critical systems, e.g., for face recognition, autonomous car control and malware detection. It is also shown that DNNs are subject to attacks such as adversarial perturbation and thus must be properly tested. Many coverage criteria for DNN since have been proposed, inspired by the success of code coverage criteria for software programs. The expectation is that if a DNN is a well tested (and retrained) according to such coverage criteria, it is more likely to be robust. In this work, we conduct an empirical study to evaluate the relationship between coverage, robustness and attack/defense metrics for DNN. Our study is the largest to date and systematically done based on 100 DNN models and 25 metrics. One of our findings is that there is limited correlation between coverage and robustness, i.e., improving coverage does not help improve the robustness. Our dataset and implementation have been made available to serve as a benchmark for future studies on testing DNN.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A NEW MEASURE OF RANK CORRELATION
• NOTES ON THE HISTORY OF CORRELATION
• Gradient-based learning applied to document recognition
• "General Intelligence," Objectively Determined and Measured
• End to End Learning for Self-Driving Cars
• Adversarial Examples in the Physical World
• Adversarial Machine Learning at Scale
• Parseval Networks: Improving Robustness to Adversarial Examples
• Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
• Concolic Testing for Deep Neural Networks
• Lipschitz regularity of deep neural networks: analysis and efficient estimation
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Robustness May Be at Odds with Accuracy
• TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
• TensorFlow: A system for large-scale machine learning
• Learning Multiple Layers of Features from Tiny Images

Related

• A Comprehensive Evaluation Framework for Deep Model Robustness
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis
• Revisiting Neuron Coverage Metrics and Quality of Deep Neural Networks
• Towards Understanding Deep Learning Model in Image Recognition via Coverage Test
• SoK: Certified Robustness for Deep Neural Networks