Retrieval Augmented Generation based context discovery for ASR

2025/09/23 by Siskos, Dimitrios, Papadopoulos, Stavros, Parada, Pablo Peso +3
#Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2509.19567

Abstract

This work investigates retrieval augmented generation as an efficient strategy for automatic context discovery in context-aware Automatic Speech Recognition (ASR) system, in order to improve transcription accuracy in the presence of rare or out-of-vocabulary terms. However, identifying the right context automatically remains an open challenge. This work proposes an efficient embedding-based retrieval approach for automatic context discovery in ASR. To contextualize its effectiveness, two alternatives based on large language models (LLMs) are also evaluated: (1) large language model (LLM)-based context generation via prompting, and (2) post-recognition transcript correction using LLMs. Experiments on the TED-LIUMv3, Earnings21 and SPGISpeech demonstrate that the proposed approach reduces WER by up to 17% (percentage difference) relative to using no-context, while the oracle context results in a reduction of up to 24.1%.

Citations

• Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation
• Locality enhanced dynamic biasing and sampling strategies for contextual ASR
• Retrieve and Copy: Scaling ASR Personalization to Large Catalogs
• Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition
• Contextual Adapters for Personalized Speech Recognition in Neural Transducers
• Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection
• SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Contextual RNN-T For Open Domain ASR
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Conformer: Convolution-augmented Transformer for Speech Recognition

Related

• Contextualization of ASR with LLM using phonetic retrieval-based augmentation
• Hearing More with Less: Multi-Modal Retrieval-and-Selection Augmented Conversational LLM-Based ASR
• ContextASR-Bench: A Massive Contextual Speech Recognition Benchmark
• LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation
• Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation