All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously

2018/01/31 by Aaron Fisher, Cynthia Rudin, Francesca Dominici · 4 citations
Mathematics · #stat.ME

paper · pdf

published as Journal of Machine Learning Research 20 (177), 1-81, 2019 · The final, published article is now available at http://www.jmlr.org/papers/v20/18-760.html . This version contains minor changes to the introductory sections, and some typo corrections. The title of this article changed twice during the revision process (see v1 and v3 on arxiv)

arxiv created 2019/12/23 · arxiv updated 2019/12/24

Abstract

Variable importance (VI) tools describe how much covariates contribute to a prediction model's accuracy. However, important variables for one well-performing model (for example, a linear model f(x)=x^Tβ with a fixed coefficient vector β) may be unimportant for another model. In this paper, we propose model class reliance (MCR) as the range of VI values across all well-performing model in a prespecified class. Thus, MCR gives a more comprehensive description of importance by accounting for the fact that many prediction models, possibly of different parametric forms, may fit the data well. In the process of deriving MCR, we show several informative results for permutation-based VI estimates, based on the VI measures used in Random Forests. Specifically, we derive connections between permutation importance estimates for a single prediction model, U-statistics, conditional variable importance, conditional causal effects, and linear model coefficients. We then give probabilistic bounds for MCR, using a novel, generalizable technique. We apply MCR to a public data set of Broward County criminal records to study the reliance of recidivism prediction models on sex and race. In this application, MCR can be used to help inform VI for unknown, proprietary models.

Cited by

• Comparing Post-Hoc Explainable AI Methods for Interpreting Black-Box EEG Models in Depression Detection
• ArborEnum: Decision Tree Rashomon Sets over Continuous Features
• Is Personalized Modality Weighting Actually Personalized? A Controlled Audit of Per-User Weighting Claims in Multimodal Recommenders
• GlyTwin: Digital Twin for Glucose Control in Type 1 Diabetes Through Optimal Behavioral Modifications Using Patient-Centric Counterfactuals
• A Two-Stage Interpretable Matching Framework for Causal Inference

Related

• Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models
• Decorrelated Variable Importance
• A Simple and Effective Model-Based Variable Importance Measure
• Mathematical Theory of Collinearity Effects on Machine Learning Variable Importance Measures
• Marginal and Conditional Importance Measures from Machine Learning Models and Their Relationship with Conditional Average Treatment Effect