Defending LLMs against Jailbreaking Attacks via Backtranslation

2024/02/26 by Yihan Wang, Wang, Yihan, Zhouxing Shi +6 · 2 voices · 33 citations
Computer Science · Psychology · #Business #Computer science #Computer security #Criminology #Cybercrime and Law Enforcement Studies #Digital and Cyber Forensics #Law, AI, and Intellectual Property #Political science #Psychology #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2402.16459

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/02/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conceal its harmful intent. In this paper, we propose a new method for defending LLMs against jailbreaking attacks by ``backtranslation''. Specifically, given an initial response generated by the target LLM from an input prompt, our backtranslation prompts a language model to infer an input prompt that can lead to the response. The inferred prompt is called the backtranslated prompt which tends to reveal the actual intent of the original prompt, since it is generated based on the LLM's response and not directly manipulated by the attacker. We then run the target LLM again on the backtranslated prompt, and we refuse the original prompt if the model refuses the backtranslated prompt. We explain that the proposed defense provides several benefits on its effectiveness and efficiency. We empirically demonstrate that our defense significantly outperforms the baselines, in the cases that are hard for the baselines, and our defense also has little impact on the generation quality for benign input prompts. Our implementation is based on our library for LLM jailbreaking defense algorithms at \urlhttps://github.com/YihanWang617/llm-jailbreaking-defense, and the code for reproducing our experiments is available at \urlhttps://github.com/YihanWang617/LLM-Jailbreaking-Defense-Backtranslation.

Cited by

• JavelinGuard: Low-Cost Transformer Architectures for LLM Security
• When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
• TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations
• When Safety Blocks Sense: Measuring Semantic Confusion in LLM Refusals
• The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
• SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
• DSCD: Large Language Model Detoxification with Self-Constrained Decoding
• A-IPO: Adaptive Intent-driven Preference Optimization
• Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
• Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense
• Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability
• HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
• Active Attacks: Red-teaming LLMs via Adaptive Environments
• Bidirectional Intention Inference Enhances LLMs' Defense Against Multi-Turn Jailbreak Attacks
• LoRA is All You Need for Safety Alignment of Reasoning LLMs
• CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors
• TuCo: Measuring the Contribution of Fine-Tuning to Individual Responses of LLMs
• A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures
• MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning
• From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
• IndoSafety: Culturally Grounded Safety for LLMs in Indonesian Languages
• Learning Safety Constraints for Large Language Models
• PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
• Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Models
• Concept-Level Explainability for Auditing & Steering LLM Responses
• POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs
• LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures
• Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types
• HoneyTrap: Deceiving Large Language Model Attackers to Honeypot Traps with Resilient Multi-Agent Defense
• DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification
• EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety
• Defense against Prompt Injection Attacks via Mixture of Encodings

Discussions

• Defending LLMs against Jailbreaking Attacks via Backtranslation [hn, 67 points, 48 comments]
• Defending LLMs against Jailbreaking Attacks via Backtranslation https://arxiv.org/abs/2402.16459 [bsky, 1 points, 0 comments]

Related

• Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks
• Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak
• FlexLLM: Exploring LLM Customization for Moving Target Defense on Black-Box LLMs Against Jailbreak Attacks
• Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
• LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper