Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense

2022/02/11 by Guangyu Shen, Shen, Guangyu, Yingqi Liu +13 · 10 citations
Computer Science · Medicine · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multiple Myeloma Research and Treatments #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2202.05749

arxiv created 2022/02/11 · openalex publication_date 2022/02/11 · arxiv updated 2022/02/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We develop a novel optimization method for NLPbackdoor inversion. We leverage a dynamically reducing temperature coefficient in the softmax function to provide changing loss landscapes to the optimizer such that the process gradually focuses on the ground truth trigger, which is denoted as a one-hot value in a convex hull. Our method also features a temperature rollback mechanism to step away from local optimals, exploiting the observation that local optimals can be easily deter-mined in NLP trigger inversion (while not in general optimization). We evaluate the technique on over 1600 models (with roughly half of them having injected backdoors) on 3 prevailing NLP tasks, with 4 different backdoor attacks and 7 architectures. Our results show that the technique is able to effectively and efficiently detect and remove backdoors, outperforming 4 baseline methods.

Cited by

• Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Models
• Rounding-Guided Backdoor Injection in Deep Learning Model Quantization
• Unmasking Backdoors: An Explainable Defense via Gradient-Attention Anomaly Scoring for Pre-trained Language Models
• From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs
• Inverting Trojans in LLMs
• Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm
• Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models
• Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models
• Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models
• The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers

Related

• Backdoor Attack against NLP models with Robustness-Aware Perturbation defense
• PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
• Robust Backdoor Removal by Reconstructing Trigger-Activated Changes in Latent Representation
• Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks
• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization