Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization

2023/04/24 by Mingli Zhu, Zhu, Mingli, Shaokui Wei +7 · 15 citations
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2304.11823

openalex publication_date 2023/04/24 · openalex created_date 2023/04/27 · openalex updated_date 2026/07/28

Abstract

Backdoor defense, which aims to detect or mitigate the effect of malicious triggers introduced by attackers, is becoming increasingly critical for machine learning security and integrity. Fine-tuning based on benign data is a natural defense to erase the backdoor effect in a backdoored model. However, recent studies show that, given limited benign data, vanilla fine-tuning has poor defense performance. In this work, we provide a deep study of fine-tuning the backdoored model from the neuron perspective and find that backdoorrelated neurons fail to escape the local minimum in the fine-tuning process. Inspired by observing that the backdoorrelated neurons often have larger norms, we propose FTSAM, a novel backdoor defense paradigm that aims to shrink the norms of backdoor-related neurons by incorporating sharpness-aware minimization with fine-tuning. We demonstrate the effectiveness of our method on several benchmark datasets and network architectures, where it achieves state-of-the-art defense performance. Overall, our work provides a promising avenue for improving the robustness of machine learning models against backdoor attacks.

Cited by

• Anti-Backdoor Coreset Selection via Cumulative Entropy
• The Eminence in Shadow: Exploiting Feature Boundary Ambiguity for Robust Backdoor Attacks
• Breaking the Stealth-Potency Trade-off in Clean-Image Backdoors with Generative Trigger Optimization
• Backdoor Attacks Against Speech Language Models
• Backdoor Mitigation via Invertible Pruning Masks
• NT-ML: Backdoor Defense via Non-target Label Training and Mutual Learning
• Proactive Disentangled Modeling of Trigger-Object Pairings for Backdoor Defense
• AutoDebias: Automated Framework for Debiasing Text-to-Image Models
• BURN: Backdoor Unlearning via Adversarial Boundary Analysis
• Circumventing Backdoor Space via Weight Symmetry
• MergeGuard: Efficient Thwarting of Trojan Attacks in Machine Learning Models
• Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies
• A Survey on Privacy Risks and Protection in Large Language Models
• Secure Transfer Learning: Training Clean Models Against Backdoor in (Both) Pre-trained Encoders and Downstream Datasets
• Exploring Backdoor Attack and Defense for LLM-empowered Recommendations

Related

• Fine-Tuning Is All You Need to Mitigate Backdoor Attacks
• Robust Backdoor Removal by Reconstructing Trigger-Activated Changes in Latent Representation
• Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness
• Defense against Backdoor Attacks via Identifying and Purifying Bad Neurons
• BAN: Detecting Backdoors Activated by Adversarial Neuron Noise