2018/05/25 by Wen Zhang, Zhang, Wen, Jiawei Hu +5
Chemistry · Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Chemistry #Computation and Language (cs.CL) #Computer science #Data mining #FOS: Computer and information sciences #Machine Learning (stat.ML) #Machine translation #Natural Language Processing Techniques #Natural language processing #Refining (metallurgy) #Relation (database) #Speech Recognition and Synthesis #Topic Modeling #Translation (biology) #cs.AI #cs.CL #stat.ML
paper · pdf · doi:10.48550/arxiv.1805.11154
12pages, 7 figures, accepted for COLING-2018. arXiv admin note: substantial text overlap with arXiv:1709.03980
openalex created_date 2017/09/25 · openalex publication_date 2018/05/25 · arxiv created 2018/09/09 · arxiv updated 2018/09/12 · openalex updated_date 2026/07/28
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step. Whereas in practice, the former information and relationship are often useful in current step. We target on solving these problems and thus introduce relation networks to learn better representations of the source. The relation networks are able to facilitate memorization capability of recurrent neural network via associating source words with each other, this would also help retain their relationships. Then the source representations and all the relations are fed into the attention component together while decoding, with the main encoder-decoder framework unchanged. Experiments on several datasets show that our method can improve the translation performance significantly over the conventional encoder-decoder model and even outperform the approach involving supervised syntactic knowledge.