Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data

2025/04/20 by Zou, Wei, Yang, Sen, Bao, Yu +3 · 2 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2504.14669

Abstract

The rise of Large Language Models (LLMs) has reshaped machine translation (MT), but multilingual MT still relies heavily on parallel data for supervised fine-tuning (SFT), facing challenges like data scarcity for low-resource languages and catastrophic forgetting. To address these issues, we propose TRANS-ZERO, a self-play framework that leverages only monolingual data and the intrinsic multilingual knowledge of LLM. TRANS-ZERO combines Genetic Monte-Carlo Tree Search (G-MCTS) with preference optimization, achieving strong translation performance that rivals supervised methods. Experiments demonstrate that this approach not only matches the performance of models trained on large-scale parallel data but also excels in non-English translation directions. Further analysis reveals that G-MCTS itself significantly enhances translation quality by exploring semantically consistent candidates through iterative translations, providing a robust foundation for the framework's succuss.

Cited by

• DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
• Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters

Related

• Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning
• How Multilingual Are Large Language Models Fine-Tuned for Translation?
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
• A Preference-driven Paradigm for Enhanced Translation with Large Language Models
• To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages