What do Neural Machine Translation Models Learn about Morphology?

2017/01/01 by Yonatan Belinkov, Nadir Durrani, Fahim Dalvi +2 · 283 citations
Computer Science · #Artificial intelligence #Artificial neural network #Character (mathematics) #Computer science #Encoder #Encoding (memory) #Granularity #Identity (music) #Language model #Linguistics #Machine translation #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Process (computing) #Programming language #Topic Modeling #Word (group theory) #cs.CL

paper · pdf · doi:10.18653/v1/p17-1080

published as ACL 55 (2017), volume 1, 861-872 · Updated decoder experiments

openalex publication_date 2017/01/01 · arxiv created 2018/10/22 · arxiv updated 2018/10/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Neural machine translation (MT) models obtain state-of-the-art performance while maintaining a simple, end-to-end architecture. However, little is known about what these models learn about source and target languages during the training process.

Citations

• Adam: A Method for Stochastic Optimization
• Character-Aware Neural Language Models
• Visualizing and Understanding Recurrent Networks
• Long Short-Term Memory
• Sequence to Sequence Learning with Neural Networks
• Neural Machine Translation by Jointly Learning to Align and Translate
• Exploring the Limits of Language Modeling
• Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Cited by

• Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems
• Grounding Representation Similarity with Statistical Testing
• Signatures of human-like processing in Transformer forward passes