Multi-Turn Jailbreaks Are Simpler Than They Seem

2025/08/11 by Yang, Xiaoxue, Jaeha Lee, Lee, Jaeha +8 · 4 citations
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2508.07646

openalex publication_date 2025/08/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

While defenses against single-turn jailbreak attacks on Large Language Models (LLMs) have improved significantly, multi-turn jailbreaks remain a persistent vulnerability, often achieving success rates exceeding 70% against models optimized for single-turn protection. This work presents an empirical analysis of automated multi-turn jailbreak attacks across state-of-the-art models including GPT-4, Claude, and Gemini variants, using the StrongREJECT benchmark. Our findings challenge the perceived sophistication of multi-turn attacks: when accounting for the attacker's ability to learn from how models refuse harmful requests, multi-turn jailbreaking approaches are approximately equivalent to simply resampling single-turn attacks multiple times. Moreover, attack success is correlated among similar models, making it easier to jailbreak newly released ones. Additionally, for reasoning models, we find surprisingly that higher reasoning effort often leads to higher attack success rates. Our results have important implications for AI safety evaluation and the design of jailbreak-resistant systems. We release the source code at https://github.com/diogo-cruz/multi_turn_simpler

Citations

• M2S: Multi-turn to Single-turn jailbreak in Red Teaming for LLMs
• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models
• Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
• Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors
• Automated Red Teaming with GOAT: the Generative Offensive Agent Tester
• Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles
• Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
• Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
• Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue
• A StrongREJECT for Empty Jailbreaks
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
• Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search

Cited by

• Replicating TEMPEST at Scale: Multi-Turn Adversarial Attacks Against Trillion-Parameter Frontier Models
• Automating Deception: Scalable Multi-Turn LLM Jailbreaks
• LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models
• Safety and Security Analysis of Large Language Models: Benchmarking Risk Profile and Harm Potential

Related

• GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication
• A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
• Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviors
• AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models
• LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet