AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration

2025/03/20 by Andy Zhou, Kevin Wu, Zhou, Andy +17 · 18 citations
Computer Science · #Advanced Malware Detection Techniques #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Network Security and Intrusion Detection #Security and Verification in Computing

paper · pdf · doi:10.48550/arxiv.2503.15754

openalex publication_date 2025/03/20 · openalex created_date 2025/10/17 · openalex updated_date 2026/07/28

Abstract

As large language models (LLMs) become increasingly capable, security and safety evaluation are crucial. While current red teaming approaches have made strides in assessing LLM vulnerabilities, they often rely heavily on human input and lack comprehensive coverage of emerging attack vectors. This paper introduces AutoRedTeamer, a novel framework for fully automated, end-to-end red teaming against LLMs. AutoRedTeamer combines a multi-agent architecture with a memory-guided attack selection mechanism to enable continuous discovery and integration of new attack vectors. The dual-agent framework consists of a red teaming agent that can operate from high-level risk categories alone to generate and execute test cases and a strategy proposer agent that autonomously discovers and implements new attacks by analyzing recent research. This modular design allows AutoRedTeamer to adapt to emerging threats while maintaining strong performance on existing attack vectors. We demonstrate AutoRedTeamer's effectiveness across diverse evaluation settings, achieving 20% higher attack success rates on HarmBench against Llama-3.1-70B while reducing computational costs by 46% compared to existing approaches. AutoRedTeamer also matches the diversity of human-curated benchmarks in generating test cases, providing a comprehensive, scalable, and continuously evolving framework for evaluating the security of AI systems.

Cited by

• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search
• Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMs
• Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges
• Red-teaming Activation Probes using Prompted LLMs
• SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning
• PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits
• When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents?
• SafeEvalAgent: Toward Agentic and Self-Evolving Safety Evaluation of LLMs
• PolicyPad: Collaborative Prototyping of LLM Policies
• A Biosecurity Agent for Lifecycle LLM Biosecurity Alignment
• SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models
• GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity Guidance
• Security Concerns for Large Language Models: A Survey
• Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming
• ADR: An Agentic Detection System for Enterprise Agentic AI Security
• Auto-ART: Structured Literature Synthesis and Automated Adversarial Robustness Testing
• Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
• A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

Related

• Automated Progressive Red Teaming
• Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
• AgenticRed: Evolving Agentic Systems for Red-Teaming
• The Automation Advantage in AI Red Teaming
• Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming