Position: Adversarial ML for LLMs Is Not Making Any Progress

2025/02/04 by Javier Rando, Rando, Javier, Jie Zhang +5 · 3 voices · 10 citations
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.CR #cs.LG

paper · pdf · doi:10.48550/arxiv.2502.02260

openalex publication_date 2025/02/04 · arxiv published 2025/02/04 · openalex created_date 2025/10/10 · arxiv updated 2026/06/02 · openalex updated_date 2026/07/28

Abstract

In the past decade, considerable research effort has been devoted to securing machine learning (ML) models that operate in adversarial settings. Yet, progress has been slow even for simple "toy" problems (e.g., robustness to small adversarial perturbations) and is often hindered by non-rigorous evaluations. Today, adversarial ML research has shifted towards studying larger, general-purpose language models. In this position paper, we argue that the situation is now even worse: in the era of LLMs, the field of adversarial ML studies problems that are (1) less clearly defined, (2) harder to solve, and (3) even more challenging to evaluate. As a result, we caution that yet another decade of work on adversarial ML may be failing to produce meaningful progress.

Cited by

• Provably Safe Model Updates
• AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research
• One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
• When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Review
• Keep It Real: Challenges in Attacking Compression-Based Adversarial Purification
• Shaping capabilities with token-level data filtering
• Benchmarking Misuse Mitigation Against Covert Adversaries
• A Red Teaming Roadmap Towards System-Level Safety
• Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
• Intent Laundering: AI Safety Datasets Are Not What They Seem

Discussions

• Great paper on how research in adversarial #ML and #AI safety has stalled for current-gen LLMs: https://arxiv.org/abs/2502.02260 #Research #AISafety #LLM #Security [bsky, 2 points, 0 comments]
• We really hope this analysis can help the community better understand where we come from, where we stand, and what things may help us make meaningful progress in the future. Co-authored with @jiezhan [bsky, 1 points, 0 comments]
• Sorry, maar dit is niet "het AI-nieuws". Dat zou bijvoorbeeld de aankondiging van Copilot Cowork zijn, EvoPhys-World, dat Palantir in Frankrijk wordt vervangen terwijl het in Oekraïne succes boekt, eu [bsky, 1 points, 1 comments]

Related

• Unrestricted Adversarial Examples
• "Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice
• Taking off the Rose-Tinted Glasses: A Critical Look at Adversarial ML Through the Lens of Evasion Attacks
• A LLM Assisted Exploitation of AI-Guardian
• You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks