[WIP] Jailbreak Paradox: The Achilles' Heel of LLMs

2024/06/18 by Rao, Abhinav, Choudhury, Monojit, Aditya, Somak · 2 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2406.12702

Abstract

We introduce two paradoxes concerning jailbreak of foundation models: First, it is impossible to construct a perfect jailbreak classifier, and second, a weaker model cannot consistently detect whether a stronger (in a pareto-dominant sense) model is jailbroken or not. We provide formal proofs for these paradoxes and a short case study on Llama and GPT4-o to demonstrate this. We discuss broader theoretical and practical repercussions of these results.

Cited by

• CourtGuard: A Local, Multiagent Prompt Injection Classifier
• A Real-Time, Self-Tuning Moderator Framework for Adversarial Prompt Detection

Related

• Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
• Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
• You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense
• What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks
• LLM Jailbreak Detection for (Almost) Free!