Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation

2021/03/03 by Zhan, Runzhe, Liu, Xuebo, Wong, Derek F. +1 · 1 citation
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2103.02262

Abstract

Meta-learning has been sufficiently validated to be beneficial for low-resource neural machine translation (NMT). However, we find that meta-trained NMT fails to improve the translation performance of the domain unseen at the meta-training stage. In this paper, we aim to alleviate this issue by proposing a novel meta-curriculum learning for domain adaptation in NMT. During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early, and finally learns the curricula of individualities to improve the model robustness for learning domain-specific knowledge. Experimental results on 10 different low-resource domains show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains. All the codes and data are freely available at https://github.com/NLP2CT/Meta-Curriculum.

Cited by

• Readability-Robust Code Summarization via Meta Curriculum Learning

Related

• Improving Both Domain Robustness and Domain Adaptability in Machine Translation
• Curriculum Learning for Domain Adaptation in Neural Machine Translation
• Meta-Learning for Few-Shot NMT Adaptation
• Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning
• Learning a Multi-Domain Curriculum for Neural Machine Translation