Exploring Generative Error Correction for Dysarthric Speech Recognition

2025/05/26 by Moreno La Quatra, La Quatra, Moreno, Alkis Koudounas +5 · 2 citations
Computer Science · Medicine · Psychology · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Phonetics and Phonology Research #Speech Recognition and Synthesis #Voice and Speech Disorders #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.20163

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Despite the remarkable progress in end-to-end Automatic Speech Recognition (ASR) engines, accurately transcribing dysarthric speech remains a major challenge. In this work, we proposed a two-stage framework for the Speech Accessibility Project Challenge at INTERSPEECH 2025, which combines cutting-edge speech recognition models with LLM-based generative error correction (GER). We assess different configurations of model scales and training strategies, incorporating specific hypothesis selection to improve transcription accuracy. Experiments on the Speech Accessibility Project dataset demonstrate the strength of our approach on structured and spontaneous speech, while highlighting challenges in single-word recognition. Through comprehensive analysis, we provide insights into the complementary roles of acoustic and linguistic modeling in dysarthric speech recognition

Citations

• Bilingual Dual-Head Deep Model for Parkinson's Disease Detection from Speech
• voc2vec: A Foundation Model for Non-Verbal Vocalization
• CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions
• Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond
• Exploiting Foundation Models and Speech Enhancement for Parkinson's Disease Detection from Speech in Real-World Operative Conditions
• Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesis
• Enhancing Pre-trained ASR System Fine-tuning for Dysarthric Speech Recognition using Adversarial Data Augmentation
• WhisperX: Time-Accurate Speech Transcription of Long-Form Audio
• Robust Speech Recognition via Large-Scale Weak Supervision
• Scaling Instruction-Finetuned Language Models
• LoRA: Low-Rank Adaptation of Large Language Models
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• SUPERB: Speech processing Universal PERformance Benchmark
• VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• pyannote.audio: neural building blocks for speaker diarization
• BERTScore: Evaluating Text Generation with BERT
• MUSAN: A Music, Speech, and Noise Corpus

Cited by

• Confidence-Guided Error Correction for Disordered Speech Recognition
• A Concept-based approach to Voice Disorder Detection

Related

• Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches
• Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
• Zero- and One-Shot Data Augmentation for Sentence-Level Dysarthric Speech Recognition in Constrained Scenarios
• Zero-Shot Recognition of Dysarthric Speech Using Commercial Automatic Speech Recognition and Multimodal Large Language Models
• A study on the impact of Self-Supervised Learning on automatic dysarthric speech assessment