Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models

2021/03/29 by Wenkai Yang, Lei Li, Yang, Wenkai +9 · 6 citations
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2103.15543

openalex publication_date 2021/03/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent studies have revealed a security threat to natural language processing (NLP) models, called the Backdoor Attack. Victim models can maintain competitive performance on clean samples while behaving abnormally on samples with a specific trigger word inserted. Previous backdoor attacking methods usually assume that attackers have a certain degree of data knowledge, either the dataset which users would use or proxy datasets for a similar task, for implementing the data poisoning procedure. However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples. Experimental results on sentiment analysis and sentence-pair classification tasks show that our method is more efficient and stealthier. We hope this work can raise the awareness of such a critical security risk hidden in the embedding layers of NLP models. Our code is available at https://github.com/lancopku/Embedding-Poisoning.

Citations

• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Input-Aware Dynamic Backdoor Attack

Cited by

• Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models
• Strategic Sample Selection for Improved Clean-Label Backdoor Attacks in Text Classification
• Pruning Strategies for Backdoor Defense in LLMs
• On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
• How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data
• BadReasoner: Planting Tunable Overthinking Backdoors into Large Reasoning Models for Fun or Profit

Related

• Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks
• Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models
• Triggerless Backdoor Attack for NLP Tasks with Clean Labels
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
• Natural Backdoor Attack on Text Data