Intention Analysis Makes LLMs A Good Jailbreak Defender

2024/01/12 by Zhang, Yuqi, Ding, Liang, Zhang, Lefei +1 · 17 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2401.06561

Abstract

Aligning large language models (LLMs) with human values, particularly when facing complex and stealthy jailbreak attacks, presents a formidable challenge. Unfortunately, existing methods often overlook this intrinsic nature of jailbreaks, which limits their effectiveness in such complex scenarios. In this study, we present a simple yet highly effective defense strategy, i.e., Intention Analysis (\mathbbIA). \mathbbIA works by triggering LLMs' inherent self-correct and improve ability through a two-stage process: 1) analyzing the essential intention of the user input, and 2) providing final policy-aligned responses based on the first round conversation. Notably, \mathbbIA is an inference-only method, thus could enhance LLM safety without compromising their helpfulness. Extensive experiments on varying jailbreak benchmarks across a wide range of LLMs show that \mathbbIA could consistently and significantly reduce the harmfulness in responses (averagely -48.2% attack success rate). Encouragingly, with our \mathbbIA, Vicuna-7B even outperforms GPT-3.5 regarding attack success rate. We empirically demonstrate that, to some extent, \mathbbIA is robust to errors in generated intentions. Further analyses reveal the underlying principle of \mathbbIA: suppressing LLM's tendency to follow jailbreak prompts, thereby enhancing safety.

Cited by

• When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs
• Beyond Context: Large Language Models' Failure to Grasp Users' Intent
• Quantifying Return on Security Controls in LLM Systems
• Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactions
• SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models
• SoK: Taxonomy and Evaluation of Prompt Security in Large Language Models
• Proactive defense against LLM Jailbreak
• SafeBehavior: Simulating Human-Like Multistage Reasoning to Mitigate Jailbreak Attacks in Large Language Models
• Retrieval-Augmented Defense: Adaptive and Controllable Jailbreak Prevention for Large Language Models
• Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
• Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
• SDGO: Self-Discrimination-Guided Optimization for Consistent Safety in Large Language Models
• Mitigating Jailbreaks with Intent-Aware LLMs
• Context Misleads LLMs: The Role of Context Filtering in Maintaining Safe Alignment of LLMs
• ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments
• Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
• CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors

Related

• Mitigating Jailbreaks with Intent-Aware LLMs
• Bidirectional Intention Inference Enhances LLMs' Defense Against Multi-Turn Jailbreak Attacks
• Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak
• Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack
• Uncovering the Persuasive Fingerprint of LLMs in Jailbreaking Attacks