Minimum Risk Training for Neural Machine Translation

2015/12/08 by Shiqi Shen, Yong Cheng, Shen, Shiqi +11 · 7 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1512.02433

Accepted for publication in Proceedings of ACL 2016

openalex publication_date 2015/12/08 · arxiv created 2016/06/15 · arxiv updated 2016/06/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is capable of optimizing model parameters directly with respect to arbitrary evaluation metrics, which are not necessarily differentiable. Experiments show that our approach achieves significant improvements over maximum likelihood estimation on a state-of-the-art neural machine translation system across various languages pairs. Transparent to architectures, our approach can be applied to more neural networks and potentially benefit more NLP tasks.

Citations

• On Using Monolingual Corpora in Neural Machine Translation
• Sequence to Sequence Learning with Neural Networks
• Sequence Level Training with Recurrent Neural Networks
• Neural Headline Generation with Sentence-wise Optimization
• Effective Approaches to Attention-based Neural Machine Translation

Cited by

• RecoReward: Recommender-Guided Multimodal Description Generation for Recommendation
• Reward Models are Metrics in a Trench Coat
• DiffGRM: Diffusion-based Generative Recommendation Model
• Drax: Speech Recognition with Discrete Flow Matching
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• Discrete Sequential Prediction of Continuous Actions for Deep RL
• DeepTrans: Deep Reasoning Translation via Reinforcement Learning

Related

• High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics
• Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation
• Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
• Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016
• Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices