vix.ing · top · new · best · stats · spec

Corpora Compared: The Case of the Swedish Gigaword & Wikipedia Corpora

2020/11/06 by Tosin Adewumi, Adewumi, Tosin P., Foteini Liwicki +3 · 1 citation
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling #Wikis in Education and Collaboration

paper · pdf · doi:10.48550/arxiv.2011.03281

openalex publication_date 2020/11/06 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

In this work, we show that the difference in performance of embeddings from differently sourced data for a given language can be due to other factors besides data size. Natural language processing (NLP) tasks usually perform better with embeddings from bigger corpora. However, broadness of covered domain and noise can play important roles. We evaluate embeddings based on two Swedish corpora: The Gigaword and Wikipedia, in analogy (intrinsic) tests and discover that the embeddings from the Wikipedia corpus generally outperform those from the Gigaword corpus, which is a bigger corpus. Downstream tests will be required to have a definite evaluation.

Cited by

Related