TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning

2025/06/11 by Mingxuan Zhang, Zhang, Mingxuan, Oubo Ma +7 · 1 citation
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Adversarial system #Backdoor #Baseline (sea) #Benchmark (surveying) #Cryptography and Security (cs.CR) #Dimension (graph theory) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Heuristic #Key (lock) #Machine Learning (cs.LG) #Reinforcement learning #Task (project management)

paper · pdf · doi:10.48550/arxiv.2506.09562

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Deep reinforcement learning (DRL) has achieved remarkable success in a wide range of sequential decision-making applications, including robotics, healthcare, smart grids, and finance. Recent studies reveal that adversaries can implant backdoors into DRL agents during the training phase. These backdoors can later be activated by specific triggers during deployment, compelling the agent to execute targeted actions and potentially leading to severe consequences, such as drone crashes or vehicle collisions. However, existing backdoor attacks utilize simplistic and heuristic trigger configurations, overlooking the critical impact of trigger design on attack effectiveness. To address this gap, we introduce TooBadRL, the first framework to systematically optimize DRL backdoor triggers across three critical aspects: injection timing, trigger dimension, and manipulation magnitude. Specifically, we first introduce a performance-aware adaptive freezing mechanism to determine the injection timing during training. Then, we formulate trigger selection as an influence attribution problem and apply Shapley value analysis to identify the most influential trigger dimension for injection. Furthermore, we propose an adversarial input synthesis method to optimize the manipulation magnitude under environmental constraints. Extensive evaluations on three DRL algorithms and nine benchmark tasks demonstrate that TooBadRL outperforms five baseline methods in terms of attack success rate while only slightly affecting normal task performance. We further evaluate potential defense strategies from detection and mitigation perspectives. We open-source our code to facilitate reproducibility and further research.

Citations

• UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning
• A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learning
• Distributed Backdoor Attacks on Federated Graph Learning and Certified Defenses
• SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents
• BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning
• Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systems
• BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets
• Training language models to follow instructions with human feedback
• Jigsaw Puzzle: Selective Backdoor Attack to Subvert Malware Classifiers
• BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning
• Highly accurate protein structure prediction with AlphaFold
• CRFL: Certifiably Robust Federated Learning against Backdoor Attacks
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• Backdoor Attacks on Self-Supervised Learning
• BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
• Can a Robot Trust You? A DRL-Based Approach to Trust-Driven Human-Guided Navigation
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Graph Backdoor
• Design of intentional backdoors in sequential models
• How To Backdoor Federated Learning
• Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
• Proximal Policy Optimization Algorithms
• A Unified Approach to Interpreting Model Predictions
• OpenAI Gym
• Asynchronous Methods for Deep Reinforcement Learning
• Trust Region Policy Optimization

Cited by

• TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models

Related

• UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learning
• BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning
• Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learning
• Adversarial Inception Backdoor Attacks against Reinforcement Learning
• Backdoors in DRL: Four Environments Focusing on In-distribution Triggers