Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learning

2021/10/09 by Guanlin Liu, Liu, Guanlin, Lifeng Lai +1 · 4 citations
Computer Science · Engineering · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Smart Grid Security and Resilience

paper · pdf · doi:10.48550/arxiv.2110.04471

openalex publication_date 2021/10/09 · openalex created_date 2021/10/25 · openalex updated_date 2026/07/28

Abstract

Due to the broad range of applications of reinforcement learning (RL), understanding the effects of adversarial attacks against RL model is essential for the safe applications of this model. Prior theoretical works on adversarial attacks against RL mainly focus on either observation poisoning attacks or environment poisoning attacks. In this paper, we introduce a new class of attacks named action poisoning attacks, where an adversary can change the action signal selected by the agent. Compared with existing attack models, the attacker's ability in the proposed action poisoning attack model is more restricted, which brings some design challenges. We study the action poisoning attack in both white-box and black-box settings. We introduce an adaptive attack scheme called LCB-H, which works for most RL agents in the black-box setting. We prove that the LCB-H attack can force any efficient RL agent, whose dynamic regret scales sublinearly with the total number of steps taken, to choose actions according to a policy selected by the attacker very frequently, with only sublinear cost. In addition, we apply LCB-H attack against a popular model-free RL algorithm: UCB-H. We show that, even in the black-box setting, by spending only logarithm cost, the proposed LCB-H attack scheme can force the UCB-H agent to choose actions according to the policy selected by the attacker very frequently.

Citations

• Robust Adversarial Reinforcement Learning
• Delving into adversarial attacks on deep policies
• Adversarial Attacks on Stochastic Bandits
• Certified Adversarial Robustness via Randomized Smoothing
• On the Convergence and Robustness of Adversarial Training
• Policy Poisoning in Batch Reinforcement Learning and Control
• Adaptive Reward-Poisoning Attacks against Reinforcement Learning
• Logarithmic Regret for Reinforcement Learning with Linear Function\n Approximation
• FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance

Cited by

• Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning
• Provably Invincible Adversarial Attacks on Reinforcement Learning Systems: A Rate-Distortion Information-Theoretic Approach
• Collaborative Shadows: Distributed Backdoor Attacks in LLM-Based Multi-Agent Systems
• Advancing Robustness in Deep Reinforcement Learning with an Ensemble Defense Approach

Related

• Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments
• Online Poisoning Attack Against Reinforcement Learning under Black-box Environments
• Adaptive Reward-Poisoning Attacks against Reinforcement Learning
• Universal Black-Box Reward Poisoning Attack against Offline Reinforcement Learning
• Black-Box Targeted Reward Poisoning Attack Against Online Deep Reinforcement Learning