Incorporating Structural Alignment Biases into an Attentional Neural\n Translation Model

2016/01/06 by Trevor Cohn, Cong Duy Vu Hoang, Cohn, Trevor +9
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1601.01085

openalex publication_date 2016/01/06 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28

Abstract

Neural encoder-decoder models of machine translation have achieved impressive\nresults, rivalling traditional translation models. However their modelling\nformulation is overly simplistic, and omits several key inductive biases built\ninto traditional models. In this paper we extend the attentional neural\ntranslation model to include structural biases from word based alignment\nmodels, including positional bias, Markov conditioning, fertility and agreement\nover translation directions. We show improvements over a baseline attentional\nmodel and standard phrase-based model over several language pairs, evaluating\non difficult languages in a low resource setting.\n

Related

• Adding Interpretable Attention to Neural Translation Models Improves Word Alignment
• Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
• Advancing Explainability in Neural Machine Translation: Analytical Metrics for Attention and Alignment Consistency
• Supervised Attentions for Neural Machine Translation
• Neural Machine Translation with Supervised Attention