Improving Translation Quality by Selecting Better Data for LLM Fine-Tuning: A Comparative Analysis

2025/12/12 by de Mello, Felipe Ribeiro Fujita, Takada, Hideyuki
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Text Readability and Simplification

paper · doi:10.48550/arxiv.2512.11388

Abstract

We investigated the impact of data selection on machine translation fine-tuning for open LLMs. Using Japanese-English corpora, we compare five selectors: TF-IDF, COMET Kiwi, QuRate, FD-Score, and random selection, under controlled training conditions. We observed that semantic selectors consistently outperform lexical and geometry-based heuristics, and that even when the selected data differ by less than 3%, the impact on model performance is substantial, underscoring the sensitivity of fine-tuning to data quality.

Citations

• Exploring Parameter-Efficient Fine-Tuning and Backtranslation for the WMT 25 General Translation Task
• Qwen3 Technical Report
• Predictive Data Selection: The Data That Predicts Is the Data That Teaches
• WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects
• Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding
• Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts
• Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?
• Gemma: Open Models Based on Gemini Research and Technology
• A Survey on Data Selection for Language Models
• QuRating: Selecting High-Quality Data for Training Language Models
• Mistral 7B
• A Comparative Study on TF-IDF feature Weighting Method and its Analysis\n using Unstructured Dataset
• LLaMA: Open and Efficient Foundation Language Models
• Data Selection for Language Models via Importance Resampling
• CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
• LoRA: Low-Rank Adaptation of Large Language Models
• An Empirical Study of Example Forgetting during Deep Neural Network Learning

Related

• The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities
• Dynamic Jointly Batch Selection for Data Efficient Machine Translation Fine-Tuning
• Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?
• Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
• How Much Data is Enough Data? Fine-Tuning Large Language Models for In-House Translation: Performance Evaluation Across Multiple Dataset Sizes