A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation

2016/03/19 by Chung, Junyoung, Cho, Kyunghyun, Bengio, Yoshua · 1 citation
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.1603.06147

Abstract

The existing machine translation systems, whether phrase-based or neural, have relied almost exclusively on word-level modelling with explicit segmentation. In this paper, we ask a fundamental question: can neural machine translation generate a character sequence without any explicit segmentation? To answer this question, we evaluate an attention-based encoder-decoder with a subword-level encoder and a character-level decoder on four language pairs--En-Cs, En-De, En-Ru and En-Fi-- using the parallel corpora from WMT'15. Our experiments show that the models with a character-level decoder outperform the ones with a subword-level decoder on all of the four language pairs. Furthermore, the ensembles of neural models with a character-level decoder outperform the state-of-the-art non-neural machine translation systems on En-Cs, En-De and En-Fi and perform comparably on En-Ru.

Cited by

• FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification

Related

• Fully Character-Level Neural Machine Translation without Explicit\n Segmentation
• An Efficient Character-Level Neural Machine Translation
• Character-based Neural Machine Translation
• Neural Machine Translation with Characters and Hierarchical Encoding
• Character-Aware Decoder for Translation into Morphologically Rich Languages