LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in\n Recurrent Neural Networks

2016/06/23 by Hendrik Strobelt, Strobelt, Hendrik, Sebastian Gehrmann +5 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Data Visualization and Analytics #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE) #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.1606.07461

openalex publication_date 2016/06/23 · openalex created_date 2022/10/05 · openalex updated_date 2026/07/28

Abstract

Recurrent neural networks, and in particular long short-term memory (LSTM)\nnetworks, are a remarkably effective tool for sequence modeling that learn a\ndense black-box hidden representation of their sequential input. Researchers\ninterested in better understanding these models have studied the changes in\nhidden state representations over time and noticed some interpretable patterns\nbut also significant noise. In this work, we present LSTMVIS, a visual analysis\ntool for recurrent neural networks with a focus on understanding these hidden\nstate dynamics. The tool allows users to select a hypothesis input range to\nfocus on local state changes, to match these states changes to similar patterns\nin a large data set, and to align these results with structural annotations\nfrom their domain. We show several use cases of the tool for analyzing specific\nhidden state properties on dataset containing nesting, phrase structure, and\nchord progressions, and demonstrate how the tool can be used to isolate\npatterns for further statistical analysis. We characterize the domain, the\ndifferent stakeholders, and their goals and tasks.\n

Citations

• Recurrent Neural Network Regularization
• Finding Structure in Time
• Neural Machine Translation by Jointly Learning to Align and Translate
• rnn : Recurrent Library for Torch
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• CafGa: Customizing Feature Attributions to Explain Language Models
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• XAlgo: a Design Probe of Explaining Algorithms' Internal States via Question-Answering

Related

• Understanding Recurrent Neural State Using Memory Signatures
• Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
• Understanding Hidden Memories of Recurrent Neural Networks
• Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks
• Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks