How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation

2018/02/02 by Menaka Narayanan, Emily Chen, Narayanan, Menaka +10 · 1 voice · 10 citations
Computer Science · Materials Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning and Data Classification #Machine Learning in Materials Science #cs.AI

paper · pdf · doi:10.48550/arxiv.1802.00682

arxiv created 2018/02/02 · openalex publication_date 2018/02/02 · arxiv published 2018/02/02 · arxiv updated 2018/02/05 · openalex created_date 2022/09/30 · openalex updated_date 2026/07/28

Abstract

Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations interpretable in the specific context of verification. Suppose we have a machine learning system that predicts X, and we provide rationale for this prediction X. Given an input, an explanation, and an output, is the output consistent with the input and the supposed rationale? Via a series of user-studies, we identify what kinds of increases in complexity have the greatest effect on the time it takes for humans to verify the rationale, and which seem relatively insensitive.

Citations

• Explaining Recommendations: Design and Evaluation
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• Programs as Black-Box Explanations
• Towards A Rigorous Science of Interpretable Machine Learning
• SmoothGrad: removing noise by adding noise
• The Promise and Peril of Human Evaluation for Model Interpretability

Cited by

• Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces
• On the notion of missingness for path attribution explainability methods in medical settings: Guiding the selection of medically meaningful baselines
• Human-Centered Explainability in Interactive Information Systems: A Survey
• See What I Mean? CUE: A Cognitive Model of Understanding Explanations
• Towards Explainable Fact Checking
• Do Explanations make VQA Models more Predictable to a Human?
• An Evaluation of the Human-Interpretability of Explanation
• LIME-LLM: Probing Models with Fluent Counterfactuals, Not Broken Text
• The Impact of Explanations on Layperson Trust in Artificial Intelligence-Driven Symptom Checker Apps: Experimental Study
• Generating User-friendly Explanations for Loan Denials using GANs

Discussions

• How Do Humans Understand Explanations from Machine Learning Systems? [hn, 2 points, 0 comments]

Related

• An Evaluation of the Human-Interpretability of Explanation
• What Makes a Good Explanation?: A Harmonized View of Properties of Explanations
• From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence
• Machine Explanations and Human Understanding
• Explaining Explanations: An Overview of Interpretability of Machine Learning