2021/08/07 by Alexandr Nikolich, Nikolich, Alexandr, Arina Puchkova +1 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Artificial intelligence #Automatic summarization #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #Computer science #Core (optical fiber) #FOS: Computer and information sciences #Function (biology) #Hyperparameter #Information retrieval #Natural Language Processing Techniques #Natural language processing #Programming language #Set (abstract data type) #State (computer science) #Task (project management) #Topic Modeling #Variety (cybernetics) #cs.CL
paper · pdf · doi:10.48550/arxiv.2108.03502
arxiv created 2021/08/07 · openalex publication_date 2021/08/07 · arxiv updated 2021/08/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Automatic summarization techniques aim to shorten and generalize information given in the text while preserving its core message and the most relevant ideas. This task can be approached and treated with a variety of methods, however, not many attempts have been made to produce solutions specifically for the Russian language despite existing localizations of the state-of-the-art models. In this paper, we aim to showcase ruGPT3 ability to summarize texts, fine-tuning it on the corpora of Russian news with their corresponding human-generated summaries. Additionally, we employ hyperparameter tuning so that the model's output becomes less random and more tied to the original text. We evaluate the resulting texts with a set of metrics, showing that our solution can surpass the state-of-the-art model's performance without additional changes in architecture or loss function. Despite being able to produce sensible summaries, our model still suffers from a number of flaws, namely, it is prone to altering Named Entities present in the original text (such as surnames, places, dates), deviating from facts stated in the given document, and repeating the information in the summary.