NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks

2025/11/14 by Lama Sleem, Sleem, Lama, Jérôme François +9
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Network Security and Intrusion Detection

paper · pdf · doi:10.48550/arxiv.2511.11784

openalex publication_date 2025/11/14 · openalex created_date 2025/11/19 · openalex updated_date 2026/07/28

Abstract

Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To address these challenges without relying on threshold calibration or model fine-tuning, this work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns. Building on this insight, a novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors. It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection. Experimental results show that the proposed method consistently achieves top-tier performance, ranking first or second in accuracy across diverse models using the crafted dataset, while competing approaches exhibit notable sensitivity to model and data variations.

Citations

• Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
• JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
• Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing
• Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
• Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Framework
• Jailbreaking Black Box Large Language Models in Twenty Queries
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
• Detecting Language Model Attacks with Perplexity
• LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked
• Code of "Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs"
• This is not correct! Negation-aware Evaluation of Language Generation Systems
• A Survey of Large Language Models
• GPT-4 Technical Report
• LLaMA: Open and Efficient Foundation Language Models
• Ignore Previous Prompt: Attack Techniques For Language Models
• Language Models are Few-Shot Learners
• Sentence Length
• Efficient Formal Safety Analysis of Neural Networks
• On Detecting Adversarial Perturbations
• JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks

Related

• Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
• The bitter lesson of misuse detection
• JailbreaksOverTime: Detecting Jailbreak Attacks Under Distribution Shift
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
• Guarding the Guardrails: A Taxonomy-Driven Approach to Jailbreak Detection