PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models

2023/10/19 by Hongwei Yao, Yao, Hongwei, Jian Lou +3 · 8 citations
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.12439

openalex publication_date 2023/10/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Prompts have significantly improved the performance of pretrained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model's normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present POISONPROMPT, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of POISONPROMPT through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.

Cited by

• Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Models
• Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustness
• Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models
• LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
• Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs
• Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
• Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment
• Provably Secure Retrieval-Augmented Generation

Related

• A Survey of Recent Backdoor Attacks and Defenses in Large Language Models
• Composite Backdoor Attacks Against Large Language Models
• Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models
• P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs
• Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks