2021/09/09 by Sahana Ramnath, Ramnath, Sahana, Melvin Johnson +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2109.04443
openalex publication_date 2021/09/09 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Back-translation (BT) of target monolingual corpora is a widely used data\naugmentation strategy for neural machine translation (NMT), especially for\nlow-resource language pairs. To improve effectiveness of the available BT data,\nwe introduce HintedBT -- a family of techniques which provides hints (through\ntags) to the encoder and decoder. First, we propose a novel method of using\nboth high and low quality BT data by providing hints (as source tags on the\nencoder) to the model about the quality of each source-target pair. We don't\nfilter out low quality data but instead show that these hints enable the model\nto learn effectively from noisy data. Second, we address the problem of\npredicting whether a source token needs to be translated or transliterated to\nthe target language, which is common in cross-script translation tasks (i.e.,\nwhere source and target do not share the written script). For such cases, we\npropose training the model with additional hints (as target tags on the\ndecoder) that provide information about the operation required on the source\n(translation or both translation and transliteration). We conduct experiments\nand detailed analyses on standard WMT benchmarks for three cross-script\nlow/medium-resource language pairs: Hindi,Gujarati,Tamil-to-English. Our\nmethods compare favorably with five strong and well established baselines. We\nshow that using these hints, both separately and together, significantly\nimproves translation quality and leads to state-of-the-art performance in all\nthree language pairs in corresponding bilingual settings.\n