2021/05/27 by El Moatez Billah Nagoudi, Nagoudi, El Moatez Billah, AbdelRahim Elmadany +3 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2105.13573
openalex publication_date 2021/05/27 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Recent progress in neural machine translation (NMT) has made it possible to\ntranslate successfully between monolingual language pairs where large parallel\ndata exist, with pre-trained models improving performance even further.\nAlthough there exists work on translating in code-mixed settings (where one of\nthe pairs includes text from two or more languages), it is still unclear what\nrecent success in NMT and language modeling exactly means for translating\ncode-mixed text. We investigate one such context, namely MT from code-mixed\nModern Standard Arabic and Egyptian Arabic (MSAEA) into English. We develop\nmodels under different conditions, employing both (i) standard end-to-end\nsequence-to-sequence (S2S) Transformers trained from scratch and (ii)\npre-trained S2S language models (LMs). We are able to acquire reasonable\nperformance using only MSA-EN parallel data with S2S models trained from\nscratch. We also find LMs fine-tuned on data from various Arabic dialects to\nhelp the MSAEA-EN task. Our work is in the context of the Shared Task on\nMachine Translation in Code-Switching. Our best model achieves bf25.72 BLEU,\nplacing us first on the official shared task evaluation for MSAEA-EN.\n