Evaluating the Translation Performance of Large Language Models Based on Euas-20

2024/08/06 by Yan Huang, Wei Liu, Huang, Yan +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2408.03119

openalex publication_date 2024/08/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In recent years, with the rapid development of deep learning technology, large language models (LLMs) such as BERT and GPT have achieved breakthrough results in natural language processing tasks. Machine translation (MT), as one of the core tasks of natural language processing, has also benefited from the development of large language models and achieved a qualitative leap. Despite the significant progress in translation performance achieved by large language models, machine translation still faces many challenges. Therefore, in this paper, we construct the dataset Euas-20 to evaluate the performance of large language models on translation tasks, the translation ability on different languages, and the effect of pre-training data on the translation ability of LLMs for researchers and developers.

Cited by

• Performance and interpretability analysis of code generation large language models

Related

• Adapting Large Language Models for Document-Level Machine Translation
• A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models
• Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
• What do Large Language Models Need for Machine Translation Evaluation?
• How Multilingual Are Large Language Models Fine-Tuned for Translation?