Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks

2023/02/10 by Piotr Gaiński, Gaiński, Piotr, Klaudia Bałazy +1 · 1 citation
Computer Science · Social Sciences · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Misinformation and Its Impacts #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2302.05120

openalex publication_date 2023/02/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a novel gradient-based attack against transformer-based language models that searches for an adversarial example in a continuous space of token probabilities. Our algorithm mitigates the gap between adversarial loss for continuous and discrete text representations by performing multi-step quantization in a quantization-compensation loop. Experiments show that our method significantly outperforms other approaches on various natural language processing (NLP) tasks.

Cited by

• Improving Fine-Grained Emotion Detection in Text with BERT and GoEmotions

Related

• Gradient-based Adversarial Attacks against Text Transformers
• A Differentiable Language Model Adversarial Attack on Text Classifiers
• Modeling the Attack: Detecting AI-Generated Text by Quantifying Adversarial Perturbations
• Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework
• Adversarial Text Purification: A Large Language Model Approach for Defense