2019/10/15 by Duygu Ataman, Ataman, Duygu, Firat, Orhan +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1910.06753
openalex publication_date 2019/10/15 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
Neural Machine Translation (NMT) models generally perform translation using a\nfixed-size lexical vocabulary, which is an important bottleneck on their\ngeneralization capability and overall translation quality. The standard\napproach to overcome this limitation is to segment words into subword units,\ntypically using some external tools with arbitrary heuristics, resulting in\nvocabulary units not optimized for the translation task. Recent studies have\nshown that the same approach can be extended to perform NMT directly at the\nlevel of characters, which can deliver translation accuracy on-par with\nsubword-based models, on the other hand, this requires relatively deeper\nnetworks. In this paper, we propose a more computationally-efficient solution\nfor character-level NMT which implements a hierarchical decoding architecture\nwhere translations are subsequently generated at the level of words and\ncharacters. We evaluate different methods for open-vocabulary NMT in the\nmachine translation task from English into five languages with distinct\nmorphological typology, and show that the hierarchical decoding model can reach\nhigher translation accuracy than the subword-level NMT model using\nsignificantly fewer parameters, while demonstrating better capacity in learning\nlonger-distance contextual and grammatical dependencies than the standard\ncharacter-level NMT model.\n