SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarking

2025/08/21 by Xiangyang Zhu, Yuan Tian, Zhu, Xiangyang +9 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2508.15526

openalex publication_date 2025/08/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The rapid proliferation of large language models (LLMs) has intensified the requirement for reliable safety evaluation to uncover model vulnerabilities. To this end, numerous LLM safety evaluation benchmarks are proposed. However, existing benchmarks generally rely on labor-intensive manual curation, which causes excessive time and resource consumption. They also exhibit significant redundancy and limited difficulty. To alleviate these problems, we introduce SafetyFlow, the first agent-flow system designed to automate the construction of LLM safety benchmarks. SafetyFlow can automatically build a comprehensive safety benchmark in only four days without any human intervention by orchestrating seven specialized agents, significantly reducing time and resource cost. Equipped with versatile tools, the agents of SafetyFlow ensure process and cost controllability while integrating human expertise into the automatic pipeline. The final constructed dataset, SafetyFlowBench, contains 23,446 queries with low redundancy and strong discriminative power. Our contribution includes the first fully automated benchmarking pipeline and a comprehensive safety benchmark. We evaluate the safety of 49 advanced LLMs on our dataset and conduct extensive experiments to validate our efficacy and efficiency.

Citations

• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• Qwen3 Technical Report
• Large Language Model Agent: A Survey on Methodology, Applications and Challenges
• Gemma 3 Technical Report
• Muon is Scalable for LLM Training
• Multi-Agent Collaboration Mechanisms: A Survey of LLMs
• Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment
• SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types
• Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping
• The Llama 3 Herd of Models
• Does Refusal Training in LLMs Generalize to the Past Tense?
• AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
• CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion
• Flames: Benchmarking Value Alignment of LLMs in Chinese
• Safe RLHF: Safe Reinforcement Learning from Human Feedback
• Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs
• GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
• BBQ: A Hand-Built Bias Benchmark for Question Answering
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Cited by

• SafeRBench: A Comprehensive Benchmark for Safety Assessment in Large Reasoning Models
• SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models

Related

• Agent-SafetyBench: Evaluating the Safety of LLM Agents
• R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
• Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models
• A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
• Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward