BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT

2023/02/21 by Shi, Jiawen, Liu, Yixin, Zhou, Pan +1 · 17 citations
#Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2304.12298

Abstract

Recently, ChatGPT has gained significant attention in research due to its ability to interact with humans effectively. The core idea behind this model is reinforcement learning (RL) fine-tuning, a new paradigm that allows language models to align with human preferences, i.e., InstructGPT. In this study, we propose BadGPT, the first backdoor attack against RL fine-tuning in language models. By injecting a backdoor into the reward model, the language model can be compromised during the fine-tuning stage. Our initial experiments on movie reviews, i.e., IMDB, demonstrate that an attacker can manipulate the generated text through BadGPT.

Cited by

• Decision-Level Hijacking: Injecting Cognitive Bias into Large Language Models via Bit-Flip Attacks
• Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learning
• GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesis
• Backdoor-Powered Prompt Injection Attacks Nullify Defense Methods
• bi-GRPO: Bidirectional Optimization for Jailbreak Backdoor Injection on LLMs
• Localizing Malicious Outputs from CodeLLM
• A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
• Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models
• DualEdit: Mitigating Safety Fallback in LLM Backdoor Editing via Affirmation-Refusal Regulation
• Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralization
• Merge Hijacking: Backdoor Attacks to Model Merging of Large Language Models
• A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluation Methods
• Security Concerns for Large Language Models: A Survey
• A Linear Approach to Data Poisoning
• A Survey of Attacks on Large Language Models
• AI Ethics and Social Norms: Exploring ChatGPT's Capabilities From What to How
• Reverse-Engineering Model Editing on Language Models

Related

• ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger
• Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injection
• RatGPT: Turning online LLMs into Proxies for Malware Attacks
• Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
• From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads