Memory Visualization for Gated Recurrent Neural Networks in Speech Recognition

2016/09/28 by Zhiyuan Tang, Ying Shi, Tang, Zhiyuan +7 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Music and Audio Processing #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.1609.08789

openalex publication_date 2016/09/28 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

Recurrent neural networks (RNNs) have shown clear superiority in sequence modeling, particularly the ones with gated units, such as long short-term memory (LSTM) and gated recurrent unit (GRU). However, the dynamic properties behind the remarkable performance remain unclear in many applications, e.g., automatic speech recognition (ASR). This paper employs visualization techniques to study the behavior of LSTM and GRU when performing speech recognition tasks. Our experiments show some interesting patterns in the gated memory, and some of them have inspired simple yet effective modifications on the network structure. We report two of such modifications: (1) lazy cell update in LSTM, and (2) shortcut connections for residual learning. Both modifications lead to more comprehensible and powerful networks.

Citations

• Kaldi Speech Recognition Toolkit
• Visualizing and Understanding Neural Models in NLP
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• Understanding Hidden Memories of Recurrent Neural Networks

Related

• Improving speech recognition by revising gated recurrent units
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Understanding Recurrent Neural State Using Memory Signatures
• Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
• Investigating gated recurrent neural networks for speech synthesis