Massively Multilingual Neural Machine Translation

2019/02/28 by Roee Aharoni, Melvin Johnson, Aharoni, Roee +3 · 12 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1903.00089

openalex publication_date 2019/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multilingual neural machine translation (NMT) enables training a single model that supports translation from multiple source languages into multiple target languages. In this paper, we push the limits of multilingual NMT in terms of number of languages being used. We perform extensive experiments in training massively multilingual NMT models, translating up to 102 languages to and from English within a single model. We explore different setups for training such models and analyze the trade-offs between translation quality and various modeling decisions. We report results on the publicly available TED talks multilingual corpus where we show that massively multilingual many-to-many models are effective in low resource settings, outperforming the previous state-of-the-art while supporting up to 59 languages. Our experiments on a large-scale dataset with 102 languages to and from English and up to one million examples per direction also show promising results, surpassing strong bilingual baselines and encouraging future work on massively multilingual NMT.

Citations

• Sequence to Sequence Learning with Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Achieving Human Parity on Automatic Chinese to English News Translation

Cited by

• NSL-MT: Linguistically Informed Negative Samples for Efficient Machine Translation in Low-Resource Languages
• A Picture is Worth a Thousand (Correct) Captions: A Vision-Guided Judge-Corrector System for Multimodal Machine Translation
• Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasets
• Zero-Shot Performance Prediction for Probabilistic Scaling Laws
• Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
• Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation
• Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks
• Language Models are Few-Shot Learners
• In2x at WMT25 Translation Task
• On the Language Coverage Bias for Neural Machine Translation
• XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
• LLaVA-NeuMT: Selective Layer-Neuron Modulation for Efficient Multilingual Multimodal Translation

Related

• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
• Complete Multilingual Neural Machine Translation
• A Study of Multilingual Neural Machine Translation
• Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations