Speaker Adaptation for End-to-End CTC Models

2019/01/04 by Ke Li, Jinyu Li, Li, Ke +7 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music and Audio Processing #Natural Language Processing Techniques #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.1901.01239

openalex publication_date 2019/01/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose two approaches for speaker adaptation in end-to-end (E2E) automatic speech recognition systems. One is Kullback-Leibler divergence (KLD) regularization and the other is multi-task learning (MTL). Both approaches aim to address the data sparsity especially output target sparsity issue of speaker adaptation in E2E systems. The KLD regularization adapts a model by forcing the output distribution from the adapted model to be close to the unadapted one. The MTL utilizes a jointly trained auxiliary task to improve the performance of the main task. We investigated our approaches on E2E connectionist temporal classification (CTC) models with three different types of output units. Experiments on the Microsoft short message dictation task demonstrated that MTL outperforms KLD regularization. In particular, the MTL adaptation obtained 8.8% and 4.0% relative word error rate reductions (WERRs) for supervised and unsupervised adaptations for the word CTC model, and 9.6% and 3.8% relative WERRs for the mix-unit CTC model, respectively.

Citations

• Sequence Transduction with Recurrent Neural Networks
• Listen, Attend and Spell
• Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large\n Vocabulary Speech Recognition

Cited by

• MOPSA: Mixture of Prompt-Experts Based Speaker Adaptation for Elderly Speech Recognition

Related

• Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition
• Improving CTC-based speech recognition via knowledge transferring from pre-trained language models
• Multilingual Training and Cross-lingual Adaptation on CTC-based Acoustic Model
• Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation
• Improving End-to-End Speech Recognition with Policy Learning