The Universal Personalizer: Few-Shot Dysarthric Speech Recognition via Meta-Learning

2025/09/19 by Dhruuv Agarwal, Agarwal, Dhruuv, Harry Zhang +5
Computer Science · Medicine · Psychology · #Speech Recognition and Synthesis #Voice and Speech Disorders #Phonetics and Phonology Research

paper · pdf · doi:10.48550/arxiv.2509.15516

Abstract

Personalizing dysarthric ASR is hindered by demanding enrollment collection and per-user training. We propose a hybrid meta-training method for a single model, enabling zero-shot and few-shot on-the-fly personalization via in-context learning (ICL). On Euphonia, it achieves 13.9% Word Error Rate (WER), surpassing speaker-independent baselines (17.5%). On SAP Test-1, our 5.3% WER outperforms the challenge-winning team (5.97%). On Test-2, our 9.49% trails only the winner (8.11%) but without relying on techniques like offline model-merging or custom audio chunking. Curation yields a 40% WER reduction using random same-speaker examples, validating active personalization. While static text curation fails to beat this baseline, oracle similarity reveals substantial headroom, highlighting dynamic acoustic retrieval as the next frontier. Data ablations confirm rapid low-resource speaker adaptation, establishing the model as a practical personalized solution.

Citations

• The Interspeech 2025 Speech Accessibility Project Challenge
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• A Self-Training Approach for Whisper to Enhance Long Dysarthric Speech Recognition
• Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
• LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis
• Towards a Single ASR Model That Generalizes to Disordered Speech
• Learning to Retrieve In-Context Examples for Large Language Models
• Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
• The unreasonable effectiveness of few-shot learning for machine translation
• In-context Examples Selection for Machine Translation
• MetaICL: Learning to Learn In Context
• Personalized Automatic Speech Recognition Trained on Small Disordered Speech Datasets
• Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and\n Accented Speech
• Sequence Transduction with Recurrent Neural Networks
• AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

Related

• Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition
• Improved Dysarthric Speech to Text Conversion via TTS Personalization
• Zero- and One-Shot Data Augmentation for Sentence-Level Dysarthric Speech Recognition in Constrained Scenarios
• Cross-Learning Fine-Tuning Strategy for Dysarthric Speech Recognition Via CDSD database
• Latent Phrase Matching for Dysarthric Speech