Group Relative Policy Optimization for Speech Recognition

2025/09/02 by Prashanth Gurunath Shivakumar, Yile Gu, Shivakumar, Prashanth Gurunath +5 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #Machine Learning and Data Classification #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2509.01939

openalex publication_date 2025/09/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Speech Recognition has seen a dramatic shift towards adopting Large Language Models (LLMs). This shift is partly driven by good scalability properties demonstrated by LLMs, ability to leverage large amounts of labelled, unlabelled speech and text data, streaming capabilities with auto-regressive framework and multi-tasking with instruction following characteristics of LLMs. However, simple next-token prediction objective, typically employed with LLMs, have certain limitations in performance and challenges with hallucinations. In this paper, we propose application of Group Relative Policy Optimization (GRPO) to enable reinforcement learning from human feedback for automatic speech recognition (ASR). We design simple rule based reward functions to guide the policy updates. We demonstrate significant improvements in word error rate (upto 18.4% relative), reduction in hallucinations, increased robustness on out-of-domain datasets and effectiveness in domain adaptation.

Citations

• Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration
• DuRep: Dual-Mode Speech Representation Learning via ASR-Aware Distillation
• Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
• SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning
• F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization
• Qwen2.5-Omni Technical Report
• Understanding R1-Zero-Like Training: A Critical Perspective
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
• R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning
• Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models
• RedPajama: an Open Dataset for Training Large Language Models
• Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback
• The Llama 3 Herd of Models
• Qwen2-Audio Technical Report
• An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
• Spirit LM: Interleaved Spoken and Written Language Model
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
• SALMONN: Towards Generic Hearing Abilities for Large Language Models
• Revisiting Large Language Models as Zero-shot Relation Extractors
• LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
• Connecting Speech Encoder and Large Language Model for ASR
• Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks
• Prompting Large Language Models with Speech Recognition Abilities
• On decoder-only architecture for speech-to-text and large language model integration
• Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition
• Textually Pretrained Speech Language Models
• SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
• Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage
• Generative Spoken Language Modeling from Raw Audio
• VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
• Common Voice: A Massively-Multilingual Speech Corpus
• Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection
• Proximal Policy Optimization Algorithms
• Reinforcement Learning with a Corrupted Reward Channel

Cited by

• Thinking in cocktail party: Chain-of-Thought and reinforcement learning for target speaker automatic speech recognition

Related

• Group Relative Policy Optimization for Text-to-Speech with Large Language Models
• Explore the Reinforcement Learning for the LLM based ASR and TTS system
• Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learning
• Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection
• Customizing Speech Recognition Model with Large Language Model Feedback