One Representation per Word - Does it make Sense for Composition?

2017/02/22 by Thomas Kober, Kober, Thomas, Julie Weeds +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1702.06696

openalex publication_date 2017/02/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we investigate whether an a priori disambiguation of word senses is strictly necessary or whether the meaning of a word in context can be disambiguated through composition alone. We evaluate the performance of off-the-shelf single-vector and multi-sense vector models on a benchmark phrase similarity task and a novel task for word-sense discrimination. We find that single-sense vector models perform as well or better than multi-sense vector models despite arguably less clean elementary representations. Our findings furthermore show that simple composition functions such as pointwise addition are able to recover sense specific information from a single-sense vector model remarkably well.

Related

• A Simple and Efficient Method To Generate Word Sense Representations
• From Word to Sense Embeddings: A Survey on Vector Representations of Meaning
• De-Conflated Semantic Representations
• What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventories
• A Simple Approach to Learn Polysemous Word Embeddings