Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

2017/03/08 by Yen-Chen Lin, Lin, Yen-Chen, Zhang-Wei Hong +9 · 35 citations
Computer Science · Mathematics · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Reinforcement Learning in Robotics #cs.CR #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1703.06748

To Appear at IJCAI 2017. Project website: http://yenchenlin.me/adversarial_attack_RL/

arxiv created 2019/11/13 · arxiv updated 2019/11/14

Abstract

We introduce two tactics to attack agents trained by deep reinforcement learning algorithms using adversarial examples, namely the strategically-timed attack and the enchanting attack. In the strategically-timed attack, the adversary aims at minimizing the agent's reward by only attacking the agent at a small subset of time steps in an episode. Limiting the attack activity to this subset helps prevent detection of the attack by the agent. We propose a novel method to determine when an adversarial example should be crafted and applied. In the enchanting attack, the adversary aims at luring the agent to a designated target state. This is achieved by combining a generative model and a planning algorithm: while the generative model predicts the future states, the planning algorithm generates a preferred sequence of actions for luring the agent. A sequence of adversarial examples is then crafted to lure the agent to take the preferred sequence of actions. We apply the two tactics to the agents trained by the state-of-the-art deep reinforcement learning algorithm including DQN and A3C. In 5 Atari games, our strategically timed attack reduces as much reward as the uniform attack (i.e., attacking at every time step) does by attacking the agent 4 times less often. Our enchanting attack lures the agent toward designated target states with a more than 70% success rate. Videos are available at http://yenchenlin.me/adversarial_attack_RL/

Cited by

• Online Robust Reinforcement Learning with General Function Approximation
• Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning
• On the Tension Between Optimality and Adversarial Robustness in Policy Optimization
• Dataset Poisoning Attacks on Behavioral Cloning Policies
• Preventing Imitation Learning with Adversarial Policy Ensembles
• Certified Defenses for Data Poisoning Attacks
• Adversarial Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Model Uncertainty
• Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
• Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning
• Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning
• Neutral Agent-based Adversarial Policy Learning against Deep Reinforcement Learning in Multi-party Open Systems
• Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
• Fooling Vision and Language Models Despite Localization and Attention Mechanism
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary
• Adversarial Attack Generation Empowered by Min-Max Optimization
• Adversarial Attacks on Reinforcement Learning-based Medical Questionnaire Systems: Input-level Perturbation Strategies and Medical Constraint Validation
• ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learning
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
• On the Robustness of Cooperative Multi-Agent Reinforcement Learning
• Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Adversarial Attacks on Reinforcement Learning based Energy Management Systems of Extended Range Electric Delivery Vehicles
• Design of intentional backdoors in sequential models
• Robustness to Adversarial Attacks in Learning-Enabled Controllers
• Towards Robust Deep Reinforcement Learning against Environmental State Perturbation
• Reinforcement Learning with Perturbed Rewards
• Off-Policy Actor-Critic for Adversarial Observation Robustness: Virtual Alternative Training via Symmetric Policy Evaluation
• Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices
• Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks
• Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning
• Spatiotemporal Attacks for Embodied Agents
• Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
• Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation
• Two Can Play That Game: An Adversarial Evaluation of a Cyber-alert Inspection System
• Security Matters: A Survey on Adversarial Machine Learning
• Neural Fidelity Calibration for Informative Sim-to-Real Adaptation

Related

• Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning
• Deep-Attack over the Deep Reinforcement Learning
• Attacking and Defending Deep Reinforcement Learning Policies
• PGN: A perturbation generation network against deep reinforcement learning
• Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation