Feature Importance Explanations for Temporal Black-Box Models

2021/02/23 by Sood, Akshay, Mark Craven, Craven, Mark · 2 citations
Computer Science · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2102.11934

openalex publication_date 2021/02/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Models in the supervised learning framework may capture rich and complex representations over the features that are hard for humans to interpret. Existing methods to explain such models are often specific to architectures and data where the features do not have a time-varying component. In this work, we propose TIME, a method to explain models that are inherently temporal in nature. Our approach (i) uses a model-agnostic permutation-based approach to analyze global feature importance, (ii) identifies the importance of salient features with respect to their temporal ordering as well as localized windows of influence, and (iii) uses hypothesis testing to provide statistical rigor.

Cited by

• HyMaTE: A Hybrid Mamba and Transformer Model for EHR Representation Learning
• OrdShap: Feature Position Importance for Sequential Black-Box Models

Related

• Temporal Dependencies in Feature Importance for Time Series Predictions
• What went wrong and when? Instance-wise Feature Importance for Time-series Models
• Explaining Time Series Predictions with Dynamic Masks
• Learning Perturbations to Explain Time Series Predictions
• Priors in Time: Missing Inductive Biases for Language Model Interpretability