2021/03/13 by Jitin Krishnan, Krishnan, Jitin, Antonios Anastasopoulos +6 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Sentiment Analysis and Opinion Mining #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2103.07792
openalex publication_date 2021/03/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Predicting user intent and detecting the corresponding slots from text are\ntwo key problems in Natural Language Understanding (NLU). In the context of\nzero-shot learning, this task is typically approached by either using\nrepresentations from pre-trained multilingual transformers such as mBERT, or by\nmachine translating the source data into the known target language and then\nfine-tuning. Our work focuses on a particular scenario where the target\nlanguage is unknown during training. To this goal, we propose a novel method to\naugment the monolingual source data using multilingual code-switching via\nrandom translations to enhance a transformer's language neutrality when\nfine-tuning it for a downstream task. This method also helps discover novel\ninsights on how code-switching with different language families around the\nworld impact the performance on the target language. Experiments on the\nbenchmark dataset of MultiATIS++ yielded an average improvement of +4.2% in\naccuracy for intent task and +1.8% in F1 for slot task using our method over\nthe state-of-the-art across 8 different languages. Furthermore, we present an\napplication of our method for crisis informatics using a new human-annotated\ntweet dataset of slot filling in English and Haitian Creole, collected during\nHaiti earthquake disaster.\n