Joint decoding method for controllable contextual speech recognition based on Speech LLM

2025/08/12 by Fang, Yangui, Jing Peng, Xi Yu +12 · 1 citation
Computer Science · Psychology · #Audio and Speech Processing (eess.AS) #Emotion and Mood Recognition #FOS: Electrical engineering #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2508.08585

openalex publication_date 2025/08/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Contextual speech recognition refers to the ability to identify preferences for specific content based on contextual information. Recently, leveraging the contextual understanding capabilities of Speech LLM to achieve contextual biasing by injecting contextual information through prompts have emerged as a research hotspot.However, the direct information injection method via prompts relies on the internal attention mechanism of the model, making it impossible to explicitly control the extent of information injection. To address this limitation, we propose a joint decoding method to control the contextual information. This approach enables explicit control over the injected contextual information and achieving superior recognition performance. Additionally, Our method can also be used for sensitive word suppression recognition.Furthermore, experimental results show that even Speech LLM not pre-trained on long contextual data can acquire long contextual capabilities through our method.

Citations

• Retrieval-Augmented Speech Recognition Approach for Domain Challenges
• FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration
• SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation
• Connecting Speech Encoder and Large Language Model for ASR
• End-to-End Speech Recognition Contextualization with Large Language Models
• SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability
• Prompting Large Language Models with Speech Recognition Abilities
• N-LTP: An Open-source Neural Language Technology Platform for Chinese
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Deep context: end-to-end contextual speech recognition
• AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline
• Listen, Attend and Spell
• Sequence Transduction with Recurrent Neural Networks

Cited by

• PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognition

Related

• Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection
• CIF-based Collaborative Decoding for End-to-end Contextual Speech Recognition
• Attention-based Contextual Language Model Adaptation for Speech\n Recognition
• Efficient Long-Form Speech Recognition for General Speech In-Context Learning
• End-to-End Speech Recognition Contextualization with Large Language Models