Nearest Neighbor Machine Translation

2020/10/01 by Urvashi Khandelwal, Khandelwal, Urvashi, Angela Fan +7 · 7 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2010.00710

openalex publication_date 2020/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce k-nearest-neighbor machine translation (kNN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. kNN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results -- without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, kNN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.

Citations

• CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
• REALM: Retrieval-Augmented Language Model Pre-Training

Cited by

• Retrieval-augmented Prompt Learning for Pre-trained Foundation Models
• Mined Prompting and Metadata-Guided Generation for Wound Care Visual Question Answering
• TreePrompt: Leveraging Hierarchical Few-Shot Example Selection for Improved English-Persian and English-German Translation
• Low-Resource English-Tigrinya MT: Leveraging Multilingual Models, Custom Tokenizers, and Clean Evaluation Benchmarks
• Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation
• Case-Based Decision-Theoretic Decoding with Quality Memories
• Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins

Related

• Better Datastore, Better Translation: Generating Datastores from Pre-Trained Models for Nearest Neural Machine Translation
• Simple and Scalable Nearest Neighbor Machine Translation
• knn-seq: Efficient, Extensible kNN-MT Framework
• Fast Nearest Neighbor Machine Translation
• Nearest Neighbor Machine Translation is Meta-Optimizer on Output Projection Layer