Evaluating neural network explanation methods using hybrid documents and morphological agreement

2018/01/19 by Nina Poerner, Poerner, Nina, Benjamin Roth +3
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1801.06422

openalex publication_date 2018/01/19 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems. Both paradigms require no manual annotation and are therefore broadly applicable. We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP. We show empirically that LIMSSE, LRP and DeepLIFT are the most effective explanation methods and recommend them for explaining DNNs in NLP.

Citations

• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• Quasi-Recurrent Neural Networks

Related

• Explainability in Neural Networks for Natural Language Processing Tasks
• Explaining Deep Neural Networks
• Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling
• Explanation Methods in Deep Learning: Users, Values, Concerns and\n Challenges
• Software and application patterns for explanation methods