Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder

2020/10/06 by Alvin Chan, Yi Tay, Chan, Alvin +5
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2010.02684

openalex publication_date 2020/10/06 · openalex created_date 2020/10/15 · openalex updated_date 2026/07/28

Abstract

This paper demonstrates a fatal vulnerability in natural language inference (NLI) and text classification systems. More concretely, we present a 'backdoor poisoning' attack on NLP models. Our poisoning attack utilizes conditional adversarially regularized autoencoder (CARA) to generate poisoned training samples by poison injection in latent space. Just by adding 1% poisoned data, our experiments show that a victim BERT finetuned classifier's predictions can be steered to the poison target class with success rates of >80% when the input hypothesis is injected with the poison signature, demonstrating that NLI and text classification systems face a huge security risk.

Citations

• Semi-Supervised Learning with Deep Generative Models
• Adversarial Examples for Evaluating Reading Comprehension Systems
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Adversarially Regularized Autoencoders
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• A large annotated corpus for learning natural language inference
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Weight Poisoning Attacks on Pre-trained Models

Related

• Concealed Data Poisoning Attacks on NLP Models
• Natural Backdoor Attack on Text Data
• Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks
• BITE: Textual Backdoor Attacks with Iterative Trigger Injection
• Injecting Bias into Text Classification Models using Backdoor Attacks