Robust Physical-World Attacks on Deep Learning Models

2017/07/27 by Kevin Eykholt, Ivan Evtimov, Eykholt, Kevin +15 · 3 voices · 508 citations
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial intelligence #Computer science #Deep learning #cs.CR #cs.LG

paper · pdf · doi:10.48550/arxiv.1707.08945

published in arXiv (Cornell University) (Cornell University) · Accepted to CVPR 2018

openalex publication_date 2017/07/27 · openalex created_date 2017/10/06 · arxiv created 2018/04/10 · arxiv updated 2018/04/11 · openalex updated_date 2026/07/28

Abstract

Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations.Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm,Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. Witha perturbation in the form of only black and white stickers,we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8%of the captured video frames obtained on a moving vehicle(field test) for the target classifier.

Citations

• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• Continuous control with deep reinforcement learning
• Adversarial Manipulation of Deep Representations
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Delving into adversarial attacks on deep policies
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Rethinking the Inception Architecture for Computer Vision
• DeepFool: a simple and accurate method to fool deep neural networks
• Adversarial Examples for Semantic Segmentation and Object Detection
• Universal adversarial perturbations
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Temporal Logic-Based Multi-Vehicle Backdoor Attacks against Offline RL Agents in End-to-end Autonomous Driving
• Universal Adversarial Perturbations: A Survey
• Moving Target Defense for Deep Visual Sensing against Adversarial Examples
• On the Limitation of MagNet Defense against L₁-based Adversarial Examples
• Convergence of Adversarial Training in Overparametrized Neural Networks
• Piracy Resistant Watermarks for Deep Neural Networks
• PoTrojan: powerful neural-level trojan designs in deep learning models
• Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems
• Monocular Depth Estimators: Vulnerabilities and Attacks
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Augmenting Model Robustness with Transformation-Invariant Attacks
• Towards the first adversarially robust neural network model on MNIST
• GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments
• Incorporating Unlabeled Data into Distributionally Robust Learning
• The Human Visual System and Adversarial AI
• Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
• PhysGAN: Generating Physical-World-Resilient Adversarial Examples for Autonomous Driving
• Adversarial Examples in Remote Sensing
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
• Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking
• Standard detectors aren't (currently) fooled by physical adversarial stop signs
• Testing Robustness Against Unforeseen Adversaries
• LoRA as a Flexible Framework for Securing Large Vision Systems
• Fast Local Attack: Generating Local Adversarial Examples for Object Detectors
• Verisimilar Percept Sequences Tests for Autonomous Driving Intelligent Agent Assessment
• Efficient Preimage Approximation for Neural Network Certification
• LanCe: A Comprehensive and Lightweight CNN Defense Methodology against Physical Adversarial Attacks on Embedded Multimedia Applications
• MeshAdv: Adversarial Meshes for Visual Recognition
• Category-wise Attack: Transferable Adversarial Examples for Anchor Free Object Detection
• Causality and Generalizability: Identifiability and Learning Methods
• All You Need is "Leet": Evading Hate-speech Detection AI
• Software Engineering for AI-Based Systems: A Survey
• Adversarial Resilience Learning - Towards Systemic Vulnerability Analysis for Large and Complex Systems
• A Spectral View of Adversarially Robust Features
• Rearchitecting Classification Frameworks For Increased Robustness
• Fooling OCR Systems with Adversarial Text Images
• QEBA: Query-Efficient Boundary-Based Blackbox Attack
• Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
• Hacking Neural Networks: A Short Introduction
• AI Enabling Technologies: A Survey
• Efficient Adversarial Training with Transferable Adversarial Examples
• Adversarial Examples in RF Deep Learning: Detection of the Attack and its Physical Robustness
• Towards Adversarial Configurations for Software Product Lines
• Intermediate Level Adversarial Attack for Enhanced Transferability
• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
• Certifying Joint Adversarial Robustness for Model Ensembles
• Adversarial Robustness Study of Convolutional Neural Network for Lumbar Disk Shape Reconstruction from MR images
• Fast Object Removal Attacks on Safety-Critical Video-based Perception Systems
• Robust Semantic Segmentation with Superpixel-Mix
• One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy
• Search Space of Adversarial Perturbations against Image Filters
• The Zwicky Transient Facility: Science Objectives
• MobilBye: Attacking ADAS with Camera Spoofing
• FineFool: Fine Object Contour Attack via Attention

Discussions

• Robust Physical-World Attacks on Machine Learning Models [hn, 97 points, 14 comments]
• Robust Physical-World Attacks on Machine Learning Models [lobsters, 2 points, 0 comments]
• Robust Physical-World Attacks on Deep Learning Models (2018) [hn, 1 points, 0 comments]

Related

• Adversarial Wear and Tear: Exploiting Natural Damage for Generating Physical-World Adversarial Examples
• Physical Adversarial Examples for Object Detectors
• Snowball Adversarial Attack on Traffic Sign Classification
• Fooling the Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples Against Traffic Sign Recognition Systems
• A Survey on Physical Adversarial Attack in Computer Vision