Explaining the Attention Mechanism of End-to-End Speech Recognition Using Decision Trees

2021/10/08 by Yuanchao Wang, Wang, Yuanchao, Wenji Du +5
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Neural Networks and Applications #Sound (cs.SD) #Speech Recognition and Synthesis #Topic Modeling #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2110.03879

openalex publication_date 2021/10/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The attention mechanism has largely improved the performance of end-to-end speech recognition systems. However, the underlying behaviours of attention is not yet clearer. In this study, we use decision trees to explain how the attention mechanism impact itself in speech recognition. The results indicate that attention levels are largely impacted by their previous states rather than the encoder and decoder patterns. Additionally, the default attention mechanism seems to put more weights on closer states, but behaves poorly on modelling long-term dependencies of attention states.

Citations

• Sequence to Sequence Learning with Neural Networks
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

Related

• Attention-Based Models for Speech Recognition
• Attention Mechanism in Speaker Recognition: What Does It Learn in Deep Speaker Embedding?
• Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition
• When Can Self-Attention Be Replaced by Feed Forward Layers?
• Relaxed Attention: A Simple Method to Boost Performance of End-to-End\n Automatic Speech Recognition