Sentence Embeddings for Russian NLU

2019/10/29 by Dmitry Popov, Alexander Pugachev, Popov, Dmitry +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1910.13291

openalex publication_date 2019/10/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We investigate the performance of sentence embeddings models on several tasks for the Russian language. In our comparison, we include such tasks as multiple choice question answering, next sentence prediction, and paraphrase identification. We employ FastText embeddings as a baseline and compare it to ELMo and BERT embeddings. We conduct two series of experiments, using both unsupervised (i.e., based on similarity measure only) and supervised approaches for the tasks. Finally, we present datasets for multiple choice question answering and next sentence prediction in Russian.

Citations

• Efficient Estimation of Word Representations in Vector Space
• SentEval: An Evaluation Toolkit for Universal Sentence Representations
• Concatenated Power Mean Word Embeddings as Universal Cross-Lingual\n Sentence Representations
• Universal Sentence Encoder

Related

• The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design
• GigaEmbeddings: Efficient Russian Language Embedding Model
• Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models
• Automatically Ranked Russian Paraphrase Corpus for Text Generation
• Russian Language Datasets in the Digitial Humanities Domain and Their Evaluation with Word Embeddings