Towards the Worst-case Robustness of Large Language Models

2025/01/31 by Huanran Chen, Yinpeng Dong, Chen, Huanran +7 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2501.19040

openalex publication_date 2025/01/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent studies have revealed the vulnerability of large language models to adversarial attacks, where adversaries craft specific input sequences to induce harmful, violent, private, or incorrect outputs. In this work, we study their worst-case robustness, i.e., whether an adversarial example exists that leads to such undesirable outputs. We upper bound the worst-case robustness using stronger white-box attacks, indicating that most current deterministic defenses achieve nearly 0% worst-case robustness. We propose a general tight lower bound for randomized smoothing using fractional knapsack solvers or 0-1 knapsack solvers, and using them to bound the worst-case robustness of all stochastic defenses. Based on these solvers, we provide theoretical lower bounds for several previous empirical defenses. For example, we certify the robustness of a specific case, smoothing using a uniform kernel, against any possible attack with an average ℓ₀ perturbation of 2.02 or an average suffix length of 6.41.

Cited by

• Intellectual Property in Graph-Based Machine Learning as a Service: Attacks and Defenses

Related

• Assessing Adversarial Robustness of Large Language Models: An Empirical Study
• Safeguarding Efficacy in Large Language Models: Evaluating Resistance to Human-Written and Algorithmic Adversarial Prompts
• Differential Robustness in Transformer Language Models: Empirical Evaluation Under Adversarial Text Attacks
• Sampling-aware Adversarial Attacks Against Large Language Models
• Robustness of Large Language Models Against Adversarial Attacks