2023/10/31 by Аdilova, F.T., Davronov, R.R., Kushmuratov, S.I.
paper · doi:10.34920/ijtaidt/vol_2023_issue_3_1
Despite being spoken by nearly 50 million individuals, the Uzbek language remains underrepresented in Natural Language Processing (NLP). One primary reason is the limited availability of Uzbek linguistic resources. With the rising prominence of the Transformer architecture in NLP, it has overtaken earlier methods like convolutional and recurrent neural networks. The T5 (Text-to-Text Transfer Transformer) standardizes linguistic tasks in English by converting them into a text-to-text format. The mT5, its multilingual version, has shown promising outcomes in various NLP tasks spanning multiple languages. However, the considerable dimensions of the mT5 pose challenges for applications focused on a singular language. In our study, we fine-tuned the mT5 specifically for Uzbek, leading to a more compact T5 model. We compared this tailored model's efficiency with the mT5 on Automatic Text Summarization (ATS) and Named Entity Recognition (NER) tasks using identical protocols and datasets. Our adapted model surpassed the performance of the mT5, indicating the feasibility of developing a more compact pre-trained model with nearly half the size, without compromising results. This streamlined model also benefits from reduced memory usage, faster startup, and swifter processing times. For access to this model, please reach out.