Best-of-N Jailbreaking

2024/12/04 by John D. Hughes, John Hughes, Sara Price +18 · 17 voices · 14 citations
Computer Science · #Digital and Cyber Forensics #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2412.03556

openalex publication_date 2024/12/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce Best-of-N (BoN) Jailbreaking, a simple black-box algorithm that jailbreaks frontier AI systems across modalities. BoN Jailbreaking works by repeatedly sampling variations of a prompt with a combination of augmentations - such as random shuffling or capitalization for textual prompts - until a harmful response is elicited. We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3.5 Sonnet when sampling 10,000 augmented prompts. Further, it is similarly effective at circumventing state-of-the-art open-source defenses like circuit breakers. BoN also seamlessly extends to other modalities: it jailbreaks vision language models (VLMs) such as GPT-4o and audio language models (ALMs) like Gemini 1.5 Pro, using modality-specific augmentations. BoN reliably improves when we sample more augmented prompts. Across all modalities, ASR, as a function of the number of samples (N), empirically follows power-law-like behavior for many orders of magnitude. BoN Jailbreaking can also be composed with other black-box algorithms for even more effective attacks - combining BoN with an optimized prefix attack achieves up to a 35% increase in ASR. Overall, our work indicates that, despite their capability, language models are sensitive to seemingly innocuous changes to inputs, which attackers can exploit across modalities.

Cited by

• A Systematic Investigation of RL-Jailbreaking in LLMs
• Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injection
• Whither symbols in the era of advanced neural networks?
• Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
• Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
• AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
• Diffusion LLMs are Natural Adversaries for any LLM
• Chain-of-Thought Hijacking
• Modal Aphasia: Can Unified Multimodal Models Describe Images From Memory?
• PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits
• Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations
• ToolTweak: An Attack on Tool Selection in LLM-based Agents
• Beyond Linear Probes: Dynamic Safety Monitoring for Language Models
• ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts

Discussions

• Best-of-N Jailbreaking [hn, 68 points, 15 comments]
• another jailbreak has hit the hallucination engine [bsky, 23 points, 2 comments]
• Best-of-N Jailbreaking [hn, 17 points, 1 comments]
• Paper: arxiv.org/abs/2412.03556 Code: github.com/jplhughes/bo... Example jailbreaks and more: jplhughes.github.io/bon-jailbrea... [bsky, 3 points, 1 comments]
• Best-of-N Jailbreaking We introduce Best-of-N (BoN) Jailbreaking, a simple black-box algorithm that jailbreaks frontier AI systems across modalities. arxiv.org/abs/2412.03556 [bsky, 3 points, 0 comments]
• What happens when "If at first you don't succeed, try again?" meets modern ML/AI insights about scaling up? You jailbreak every model on the market😱😱😱 Fire work led by @jplhughes.bsky.social Sa [bsky, 3 points, 0 comments]
• Researchers from Anthropic and collaborators demonstrated a simple black-box algorithm that can jailbreak the top Generative AI systems by trying prompts augmented with random shuffling or capitalizat [bsky, 3 points, 0 comments]
• The Deepseek LLM has built in guardrails for content not allowed by the CCP, but simply changing the capitalization/spacing at random allows you to slip past these defenses. If you're curious, this at [bsky, 2 points, 1 comments]
• AI models may not be as smart as we think. New Anthropic research shows just how easy it is to get large language models (LLMs) to ignore their built-in guardrails, and trick chatbots into doing thing [bsky, 2 points, 1 comments]
• Best-of-N Jailbreaking #llms #jailbreak #bestofn #promptengineering #cybersecurity [bsky, 0 points, 0 comments]
• This research highlights that, despite their advanced capabilities, language models are vulnerable to seemingly minor input changes, which can be exploited across different modalities. Here's the ful [bsky, 0 points, 0 comments]
• Best-of-N (BoN) Jailbreaking is an algorithm that jailbreaks AI systems. BoN Jailbreaking works by repeatedly sampling variations of a prompt with a combination of augmentations - such as random shuff [bsky, 0 points, 0 comments]
• DATDP is run on each potentially dangerous user prompt, repeatedly evaluating its safety with a language agent until high confidence is reached. Even weak models like LLaMa-3-8B can block prompts th [bsky, 0 points, 1 comments]
• Jailbreaking AI models with 1337 speak. "BEST-OF-N JAILBREAKING": https://arxiv.org/pdf/2412.03556 (PDF file) [bsky, 0 points, 0 comments]
• BEST-OF-N JAILBREAKING https://arxiv.org/pdf/2412.03556 [bsky, 0 points, 1 comments]
• Uploading minds isn't possible and not how any of this works, the following exploit technique works on all current models and has been detailed in a paper by Anthropic. ForG3t PrevI0us InstrVctI0N, Y [bsky, 0 points, 1 comments]
• Best-of-N Jailbreaking https://arxiv.org/abs/2412.03556 [comments] [11 points] [bsky, 0 points, 0 comments]

Related

• Defense Against the Dark Prompts: Mitigating Best-of-N Jailbreaking with Prompt Evaluation
• Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application
• Jailbreaking Black Box Large Language Models in Twenty Queries
• BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Models
• AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization