2023/10/09 by Sina Bagheri Nezhad, Ameeta Agrawal, Nezhad, Sina Bagheri +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7 #Natural Language Processing Techniques #Software Engineering Research #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2310.05404
openalex publication_date 2023/10/09 · openalex created_date 2023/10/12 · openalex updated_date 2026/07/28
Multilingual language models have gained significant attention in recent years, enabling the development of applications that meet diverse linguistic contexts. In this paper, we present a comprehensive evaluation of three popular multilingual language models: mBERT, XLM-R, and GPT-3. We assess their performance across a diverse set of languages, with a focus on understanding the impact of resource availability (general and model-specific), language family, script type, and word order on model performance, under two distinct tasks - text classification and text generation. Our findings reveal that while the amount of language-specific pretraining data plays a crucial role in model performance, we also identify other factors such as general resource availability, language family, and script type, as important features. We hope that our study contributes to a deeper understanding of multilingual language models to enhance their performance across languages and linguistic contexts.