To MT or not to MT: An eye-tracking study on the reception by Dutch readers of different translation and creativity levels

2025/04/28 by Kyo Gerrits, Gerrits, Kyo, Ana Guerberof Arenas +1
Arts and Humanities · Psychology · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Second Language Acquisition and Learning #Subtitles and Audiovisual Media #Translation Studies and Practices

paper · pdf · doi:10.48550/arxiv.2504.19850

openalex publication_date 2025/04/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This article presents the results of a pilot study involving the reception of a fictional short story translated from English into Dutch under four conditions: machine translation (MT), post-editing (PE), human translation (HT) and original source text (ST). The aim is to understand how creativity and errors in different translation modalities affect readers, specifically regarding cognitive load. Eight participants filled in a questionnaire, read a story using an eye-tracker, and conducted a retrospective think-aloud (RTA) interview. The results show that units of creative potential (UCP) increase cognitive load and that this effect is highest for HT and lowest for MT; no effect of error was observed. Triangulating the data with RTAs leads us to hypothesize that the higher cognitive load in UCPs is linked to increases in reader enjoyment and immersion. The effect of translation creativity on cognitive load in different translation modalities at word-level is novel and opens up new avenues for further research. All the code and data are available at https://github.com/INCREC/Pilot_to_MT_or_not_to_MT

Citations

• Readers have to work harder to understand a badly translated text: an eye-tracking study into the effects of translation errors
• Quantifying the Carbon Emissions of Machine Learning
• Post-editese: an Exacerbated Translationese
• The Coding Manual for Qualitative Researchers

Related

• To be or not to be: a translation reception study of a literary text translated into Dutch and Catalan using machine translation
• Why Literary Translators should embrace Translation Technology
• Readers have to work harder to understand a badly translated text: an eye-tracking study into the effects of translation errors
• The Adverse Effects of Machine Translation on the Linguistic Skills of Human Translators
• Extending CREAMT: Leveraging Large Language Models for Literary Translation Post-Editing