vix.ing · top · new · best · stats

Distributed Representations of Sentences and Documents

2014/05/16 by Quoc V. Le, Tomas Mikolov, Tomáš Mikolov +2 · 4 voices · 5,127 citations
Computer Science · #Artificial intelligence #Bag-of-words model #Computer science #Feature (linguistics) #Feature vector #Linguistics #Natural Language Processing Techniques #Natural language processing #Paragraph #Popularity #Semantics (computer science) #Sentiment Analysis and Opinion Mining #Support vector machine #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1405.4053

published in arXiv (Cornell University) 4, 1188-1196 (Cornell University)

openalex publication_date 2014/05/16 · arxiv created 2014/05/22 · arxiv updated 2014/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. For example, "powerful," "strong" and "Paris" are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperform bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on several text classification and sentiment analysis tasks.

Citations

Cited by

Discussions

Related