Sound Explanation for Trustworthy Machine Learning

2023/06/08 by Jia, Kai, Saowakon, Pasapol, Appelbaum, Limor +1
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2306.06134

Abstract

We take a formal approach to the explainability problem of machine learning systems. We argue against the practice of interpreting black-box models via attributing scores to input components due to inherently conflicting goals of attribution-based interpretation. We prove that no attribution algorithm satisfies specificity, additivity, completeness, and baseline invariance. We then formalize the concept, sound explanation, that has been informally adopted in prior work. A sound explanation entails providing sufficient information to causally explain the predictions made by a system. Finally, we present the application of feature selection as a sound explanation for cancer prediction models to cultivate trust among clinicians.

Related

• Explainable AI needs formal notions of explanation correctness
• Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations
• Measure Utility, Gain Trust: Practical Advice for XAI Researcher
• How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
• Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions