On the Evaluation of Machine Translation for Terminology Consistency

2021/06/22 by Md Mahfuz Ibn Alam, Alam, Md Mahfuz ibn, Antonios Anastasopoulos +11
Computer Science · Biochemistry, Genetics and Molecular Biology · #Natural Language Processing Techniques #Topic Modeling #Biomedical Text Mining and Ontologies

paper · pdf · doi:10.48550/arxiv.2106.11891

Abstract

As neural machine translation (NMT) systems become an important part of professional translator pipelines, a growing body of work focuses on combining NMT with terminologies. In many scenarios and particularly in cases of domain adaptation, one expects the MT output to adhere to the constraints provided by a terminology. In this work, we propose metrics to measure the consistency of MT output with regards to a domain terminology. We perform studies on the COVID-19 domain over 5 languages, also performing terminology-targeted human evaluation. We open-source the code for computing all proposed metrics: https://github.com/mahfuzibnalam/terminology_evaluation

Related

• HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions
• Impact of Domain-Adapted Multilingual Neural Machine Translation in the Medical Domain
• Evaluating Automatic Metrics with Incremental Machine Translation Systems
• Machine Translation: A Literature Review
• It Takes Two: A Dual Stage Approach for Terminology-Aware Translation