2022/12/21 by Ife Adebara, Adebara, Ife, AbdelRahim Elmadany +5 · 2 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Text Readability and Simplification
paper · pdf · doi:10.48550/arxiv.2212.10785
Multilingual pretrained language models (mPLMs) acquire valuable, generalizable linguistic information during pretraining and have advanced the state of the art on task-specific finetuning. To date, only ~31 out of ~2,000 African languages are covered in existing language models. We ameliorate this limitation by developing SERENGETI, a massively multilingual language model that covers 517 African languages and language varieties. We evaluate our novel models on eight natural language understanding tasks across 20 datasets, comparing to 4 mPLMs that cover 4-23 African languages. SERENGETI outperforms other models on 11 datasets across the eights tasks, achieving 82.27 average F1. We also perform analyses of errors from our models, which allows us to investigate the influence of language genealogy and linguistic similarity when the models are applied under zero-shot settings. We will publicly release our models for research.\footnote\hrefhttps://github.com/UBC-NLP/serengetihttps://github.com/UBC-NLP/serengeti