Speech-Aware Neural Diarization with Encoder-Decoder Attractor Guided by Attention Constraints

2024/03/21 by PeiYing Lee, Lee, PeiYing, Hau-Yun Guo +3
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Neural Networks and Applications #Sound (cs.SD) #Speech Recognition and Synthesis #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2403.14268

openalex publication_date 2024/03/21 · openalex created_date 2024/03/24 · openalex updated_date 2026/07/28

Abstract

End-to-End Neural Diarization with Encoder-Decoder based Attractor (EEND-EDA) is an end-to-end neural model for automatic speaker segmentation and labeling. It achieves the capability to handle flexible number of speakers by estimating the number of attractors. EEND-EDA, however, struggles to accurately capture local speaker dynamics. This work proposes an auxiliary loss that aims to guide the Transformer encoders at the lower layer of EEND-EDA model to enhance the effect of self-attention modules using speaker activity information. The results evaluated on public dataset Mini LibriSpeech, demonstrates the effectiveness of the work, reducing Diarization Error Rate from 30.95% to 28.17%. We will release the source code on GitHub to allow further research and reproducibility.

Related

• Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractor
• Improving Neural Diarization through Speaker Attribute Attractors and Local Dependency Modeling
• Attention-based Encoder-Decoder End-to-End Neural Diarization with Embedding Enhancer
• Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information?
• Improving End-to-End Neural Diarization Using Conversational Summary Representations