2021/05/31 by Eleftheria Briakou, Marine Carpuat, Briakou, Eleftheria +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2105.15087
openalex publication_date 2021/05/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
While it has been shown that Neural Machine Translation (NMT) is highly\nsensitive to noisy parallel training samples, prior work treats all types of\nmismatches between source and target as noise. As a result, it remains unclear\nhow samples that are mostly equivalent but contain a small number of\nsemantically divergent tokens impact NMT training. To close this gap, we\nanalyze the impact of different types of fine-grained semantic divergences on\nTransformer models. We show that models trained on synthetic divergences output\ndegenerated text more frequently and are less confident in their predictions.\nBased on these findings, we introduce a divergent-aware NMT framework that uses\nfactors to help NMT recover from the degradation caused by naturally occurring\ndivergences, improving both translation quality and model calibration on EN-FR\ntasks.\n