Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses

2020/01/11 by Kashyap Coimbatore Murali, Murali, Kashyap Coimbatore
Computer Science · #68T35 (Primary) #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7

paper · pdf · doi:10.48550/arxiv.2001.05286

openalex publication_date 2020/01/11 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28

Abstract

In this paper, we explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) against non-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well as some possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep Neural Network, due to the regularization effect produced when training as a result of its cross task data, is more robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference). We further show that although the MT-DNN has generalized better, making it easily transferable across domains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character) the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally, we propose a domain agnostic defense which restores the model's accuracy (36.75% and 25.94% respectively) as opposed to a general-purpose defense or an off-the-shelf spell checker.

Related

• A Survey of Adversarial Defences and Robustness in NLP
• Defending against Adversarial Attack towards Deep Neural Networks via Collaborative Multi-task Training
• Enhancing Model Robustness By Incorporating Adversarial Knowledge Into Semantic Representation
• MPAT: Building Robust Deep Neural Networks against Textual Adversarial Attacks
• Adversarial Attacks and Defenses in Large Language Models: Old and New Threats