Towards Robust Neural Machine Translation

2018/05/16 by Yong Cheng, Zhaopeng Tu, Cheng, Yong +7 · 5 citations
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1805.06130

Accepted by ACL 2018

arxiv created 2018/05/16 · openalex publication_date 2018/05/16 · arxiv updated 2018/05/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation (NMT) models. In this paper, we propose to improve the robustness of NMT models with adversarial stability training. The basic idea is to make both the encoder and decoder in NMT models robust against input perturbations by enabling them to behave similarly for the original input and its perturbed counterpart. Experimental results on Chinese-English, English-German and English-French translation tasks show that our approaches can not only achieve significant improvements over strong NMT systems but also improve the robustness of NMT models.

Citations

• Improving neural networks by preventing co-adaptation of feature detectors
• Improved Techniques for Training GANs
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Neural Machine Translation with Linear Associative Unit
• Convolutional Sequence to Sequence Learning
• Improving the Robustness of Deep Neural Networks via Stability Training

Cited by

• Neural Machine Translation: A Review and Survey
• Adversarial Training with Contrastive Learning in NLP
• Simple Contrastive Representation Adversarial Learning for NLP Tasks
• A Reinforced Generation of Adversarial Examples for Neural Machine Translation
• Improving the Robustness of Speech Translation

Related

• Robust Neural Machine Translation with Doubly Adversarial Inputs
• Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?
• Evaluating Robustness to Input Perturbations for Neural Machine Translation
• Addressing the Vulnerability of NMT in Input Perturbations
• Improving Robustness in Real-World Neural Machine Translation Engines