FastInject: Injecting Unpaired Text Data into CTC-based ASR training

2023/12/14 by Deng, Keqi, Woodland, Philip C. · 1 citation
#Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2312.09100

Abstract

Recently, connectionist temporal classification (CTC)-based end-to-end (E2E) automatic speech recognition (ASR) models have achieved impressive results, especially with the development of self-supervised learning. However, E2E ASR models trained on paired speech-text data often suffer from domain shifts from training to testing. To alleviate this issue, this paper proposes a flat-start joint training method, named FastInject, which efficiently injects multi-domain unpaired text data into CTC-based ASR training. To maintain training efficiency, text units are pre-upsampled, and their representations are fed into the CTC model along with speech features. To bridge the modality gap between speech and text, an attention-based modality matching mechanism (AM3) is proposed, which retains the E2E flat-start training. Experiments show that the proposed FastInject gave a 22% relative WER reduction (WERR) for intra-domain Librispeech-100h data and 20% relative WERR on out-of-domain test sets.

Cited by

• New Insights into Optimal Alignment of Acoustic and Linguistic Representations for Knowledge Transfer in ASR

Related

• Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions
• Timestamped Embedding-Matching Acoustic-to-Word CTC ASR
• Distilling the Knowledge of BERT for CTC-based ASR
• Joint Masked CPC and CTC Training for ASR
• Improving CTC-based speech recognition via knowledge transferring from pre-trained language models