Input Validation for Neural Networks via Runtime Local Robustness Verification

2020/02/09 by Jiangchao Liu, Liu, Jiangchao, Liqian Chen +5 · 9 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Adversarial system #Algorithm #Artificial intelligence #Artificial neural network #Cardiac Arrest and Resuscitation #Computer science #Deep neural networks #F.3.1 #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Perturbation (astronomy) #Robustness (evolution)

paper · pdf · doi:10.48550/arxiv.2002.03339

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2020/02/09 · openalex created_date 2020/02/14 · openalex updated_date 2026/07/28

Abstract

Local robustness verification can verify that a neural network is robust wrt. any perturbation to a specific input within a certain distance. We call this distance Robustness Radius. We observe that the robustness radii of correctly classified inputs are much larger than that of misclassified inputs which include adversarial examples, especially those from strong adversarial attacks. Another observation is that the robustness radii of correctly classified inputs often follow a normal distribution. Based on these two observations, we propose to validate inputs for neural networks via runtime local robustness verification. Experiments show that our approach can protect neural networks from adversarial examples and improve their accuracies.

Citations

• Gradient-based learning applied to document recognition
• Anomaly detection
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Safety Verification of Deep Neural Networks
• On the (Statistical) Detection of Adversarial Examples
• Detecting Adversarial Samples from Artifacts
• Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
• Deep learning
• Adversarial Examples Are Not Bugs, They Are Features
• HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
• MagNet: a Two-Pronged Defense against Adversarial Examples

Cited by

• A New Perspective On AI Safety Through Control Theory Methodologies
• Enhancing System Self-Awareness and Trust of AI: A Case Study in Trajectory Prediction and Planning

Related

• Formal Reasoning About Confidence and Automated Verification of Neural Networks
• Neural Network Robustness as a Verification Property: A Principled Case Study
• Verification of Neural Networks' Global Robustness
• Verification of Neural Networks: Specifying Global Robustness using Generative Models
• A Formally Verified Robustness Certifier for Neural Networks (Extended Version)