Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds

2013/12/19 by Sergienya, Irina, Schütze, Hinrich
#Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.6 #I.2.7

paper · doi:10.48550/arxiv.1312.5559

Abstract

There are two main approaches to the distributed representation of words: low-dimensional deep learning embeddings and high-dimensional distributional models, in which each dimension corresponds to a context word. In this paper, we combine these two approaches by learning embeddings based on distributional-model vectors - as opposed to one-hot vectors as is standardly done in deep learning. We show that the combined approach has better performance on a word relatedness judgment task.

Related

• Word Representations via Gaussian Embedding
• Word Embeddings: A Survey
• Mapping distributional to model-theoretic semantic spaces: a baseline
• Can Network Embedding of Distributional Thesaurus be Combined with Word Vectors for Better Representation?
• Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in\n Distributional Semantic Models