The Effects of Data Size and Frequency Range on Distributional Semantic Models

2016/09/27 by Magnus Sahlgren, Sahlgren, Magnus, Alessandro Lenci +1 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Healthcare #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1609.08293

openalex publication_date 2016/09/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper investigates the effects of data size and frequency range on distributional semantic models. We compare the performance of a number of representative models for several test settings over data of varying sizes, and over test items of various frequency. Our results show that neural network-based models underperform when the data is small, and that the most reliable model over data of varying sizes and frequency ranges is the inverted factorized model.

Cited by

• Let's Simply Count: Quantifying Distributional Similarity Between Activities in Event Data

Related

• A comparative evaluation and analysis of three generations of Distributional Semantic Models
• Mapping distributional to model-theoretic semantic spaces: a baseline
• Scaling Laws for Neural Language Models
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds