Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data

2020/04/27 by Benjamin Shickel, Parisa Rashidi, Shickel, Benjamin +1
Computer Science · Mathematics · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare #Topic Modeling #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2004.12524

arxiv created 2020/04/27 · openalex publication_date 2020/04/27 · arxiv updated 2020/04/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep learning continues to revolutionize an ever-growing number of critical application areas including healthcare, transportation, finance, and basic sciences. Despite their increased predictive power, model transparency and human explainability remain a significant challenge due to the "black box" nature of modern deep learning models. In many cases the desired balance between interpretability and performance is predominately task specific. Human-centric domains such as healthcare necessitate a renewed focus on understanding how and why these frameworks are arriving at critical and potentially life-or-death decisions. Given the quantity of research and empirical successes of deep learning for computer vision, most of the existing interpretability research has focused on image processing techniques. Comparatively, less attention has been paid to interpreting deep learning frameworks using sequential data. Given recent deep learning advancements in highly sequential domains such as natural language processing and physiological signal processing, the need for deep sequential explanations is at an all-time high. In this paper, we review current techniques for interpreting deep learning techniques involving sequential data, identify similarities to non-sequential methods, and discuss current limitations and future avenues of sequential interpretability research.

Citations

• Reasoning about Entailment with Neural Attention
• Neural Machine Translation by Jointly Learning to Align and Translate
• Deep Motif: Visualizing Genomic Sequence Classifications
• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• The Mythos of Model Interpretability
• Understanding Neural Networks through Representation Erasure
• Towards A Rigorous Science of Interpretable Machine Learning
• A Structured Self-attentive Sentence Embedding
• Deep learning with convolutional neural networks for EEG decoding and visualization
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• Interpretable Deep Convolutional Neural Networks via Meta-learning
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
• Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis
• "Why Should I Trust You?": Explaining the Predictions of Any Classifier
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism

Related

• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• On Interpretability of Artificial Neural Networks: A Survey
• Applications of interpretable deep learning in neuroimaging: a comprehensive review
• Visual Interpretability for Deep Learning: a Survey
• Deep learning interpretability in neuroimaging: A comprehensive survey and methodological recommendations