Theoretical Guarantees for Minimum Bayes Risk Decoding

2025/02/18 by Ichihara, Yuki, Yuu Jinnai, Kaito Ariu +6 · 2 citations
Computer Science · Mathematics · #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #FOS: Computer and information sciences #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2502.12685

openalex publication_date 2025/02/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Minimum Bayes Risk (MBR) decoding optimizes output selection by maximizing the expected utility value of an underlying human distribution. While prior work has shown the effectiveness of MBR decoding through empirical evaluation, few studies have analytically investigated why the method is effective. As a result of our analysis, we show that, given the size n of the reference hypothesis set used in computation, MBR decoding approaches the optimal solution with high probability at a rate of O(n^-(1)/(2)), under certain assumptions, even though the language space Y is significantly larger |Y|≫ n. This result helps to theoretically explain the strong performance observed in several prior empirical studies on MBR decoding. In addition, we provide the performance gap for maximum-a-posteriori (MAP) decoding and compare it to MBR decoding. The result of this paper indicates that MBR decoding tends to converge to the optimal solution faster than MAP decoding in several cases.

Cited by

• Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition
• Case-Based Decision-Theoretic Decoding with Quality Memories

Related

• It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk
• Faster Minimum Bayes Risk Decoding with Confidence-based Pruning
• Model-Based Minimum Bayes Risk Decoding for Text Generation
• Agreement-Constrained Probabilistic Minimum Bayes Risk Decoding
• Linear-time Minimum Bayes Risk Decoding with Reference Aggregation