Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition

2025/10/22 by Yuu Jinnai, Jinnai, Yuu · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Speech and dialogue systems #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2510.19471

openalex publication_date 2025/10/22 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

Recent work has shown that sample-based Minimum Bayes Risk (MBR) decoding outperforms beam search in text-to-text generation tasks, such as machine translation, text summarization, and image captioning. On the other hand, beam search is the current practice for speech-to-text tasks such as automatic speech recognition (ASR) and Speech Translation (ST). Given that MBR decoding is effective in text-to-text generation tasks, it is reasonable to expect it to also be effective for speech-to-text tasks. In this paper, we evaluate MBR decoding for ASR and ST tasks on English and Japanese using Whisper and its derivative models. We observe that the accuracy of MBR decoding outperforms that of beam search in most of the experimental settings we have evaluated. The results show that MBR decoding is a promising method for offline ASR and ST tasks that require high accuracy. The code is available at https://github.com/CyberAgentAILab/mbr-for-asr

Citations

• MOVER: Combining Multiple Meeting Recognition Systems
• Theoretical Guarantees for Minimum Bayes Risk Decoding
• On the Robust Approximation of ASR Metrics
• Findings of the IWSLT 2024 Evaluation Campaign
• Mitigating Metric Bias in Minimum Bayes Risk Decoding
• Better Instruction-Following Through Minimum Bayes Risk
• Speech Recognition Rescoring with Large Speech-Text Foundation Models
• ProGRes: Prompted Generative Rescoring on ASR n-Best
• Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithms
• On the True Distribution Approximation of Minimum Bayes-Risk Decoding
• Centroid-Based Efficient Minimum Bayes Risk Decoding
• It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
• Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
• Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding
• Faster Minimum Bayes Risk Decoding with Confidence-based Pruning
• Model-Based Minimum Bayes Risk Decoding for Text Generation
• Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling
• Acoustic Model Fusion for End-to-end Speech Recognition
• Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
• Investigating End-to-End ASR Architectures for Long Form Audio Transcription
• Can Generative Large Language Models Perform ASR Error Correction?
• A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-Supervision
• Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation
• End-to-End Speech Recognition: A Survey
• Robust Speech Recognition via Large-Scale Weak Supervision
• Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
• Truncation Sampling as Language Model Desmoothing
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• WebGPT: Browser-assisted question-answering with human feedback
• High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics
• Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation
• FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition
• Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding
• VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation
• fairseq S2T: Fast Speech-to-Text Modeling with fairseq
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Adapting End-to-End Speech Recognition for Readable Subtitles
• Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation
• Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation
• BLEURT: Learning Robust Metrics for Text Generation
• Deliberation Model Based Two-Pass End-to-End Speech Recognition
• Common Voice: A Massively-Multilingual Speech Corpus
• A comparison of end-to-end models for long-form speech recognition
• Recognizing long-form speech using streaming end-to-end models
• Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• The Curious Case of Neural Text Degeneration
• A spelling correction model for end-to-end speech recognition
• What Doubling Tricks Can and Can't Do for Multi-Armed Bandits
• JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis
• Challenges in Data-to-Document Generation
• Attention Is All You Need
• Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning
• MUSAN: A Music, Speech, and Noise Corpus
• Sequence to Sequence Learning with Neural Networks
• Sequence Transduction with Recurrent Neural Networks

Cited by

• Zero-Shot Context-Aware ASR for Diverse Arabic Varieties

Related

• Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding
• Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation
• Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
• Faster Minimum Bayes Risk Decoding with Confidence-based Pruning
• Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation