Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling

2021/01/28 by David Harbecke, Harbecke, David
Computer Science · #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2101.11889

openalex publication_date 2021/01/28 · openalex created_date 2021/02/01 · openalex updated_date 2026/07/28

Abstract

Deep neural networks are powerful statistical learners. However, their predictions do not come with an explanation of their process. To analyze these models, explanation methods are being developed. We present a novel explanation method, called OLM, for natural language processing classifiers. This method combines occlusion and language modeling, which are techniques central to explainability and NLP, respectively. OLM gives explanations that are theoretically sound and easy to understand. We make several contributions to the theory of explanation methods. Axioms for explanation methods are an interesting theoretical concept to explore their basics and deduce methods. We introduce a new axiom, give its intuition and show it contradicts another existing axiom. Additionally, we point out theoretical difficulties of existing gradient-based and some occlusion-based explanation methods in natural language processing. We provide an extensive argument why evaluation of explanation methods is difficult. We compare OLM to other explanation methods and underline its uniqueness experimentally. Finally, we investigate corner cases of OLM and discuss its validity and possible improvements.

Citations

• Efficient Estimation of Word Representations in Vector Space
• The Loss Surfaces of Multilayer Networks
• A Stochastic Approximation Method
• THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED
• How to Explain Individual Classification Decisions
• ImageNet classification with deep convolutional neural networks
• A vector space model for automatic indexing
• European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Opening the Black Box of Deep Neural Networks via Information
• Axiomatic Attribution for Deep Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• Learning how to explain neural networks: PatternNet and PatternAttribution
• A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
• Sanity Checks for Saliency Maps
• A Survey of Methods for Explaining Black Box Models
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Full-Gradient Representation for Neural Network Visualization

Related

• Considering Likelihood in NLP Classification Explanations with Occlusion\n and Language Modeling
• Evaluating neural network explanation methods using hybrid documents and morphological agreement
• Learning Explanations from Language Data
• NILE : Natural Language Inference with Faithful Natural Language\n Explanations
• Can Language Models Explain Their Own Classification Behavior?