2011/01/24 by Daoud Clarke, Clarke, Daoud · 1 voice
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Semantic Web and Ontologies
paper · pdf · doi:10.48550/arxiv.1101.4479
Techniques in which words are represented as vectors have proved useful in\nmany applications in computational linguistics, however there is currently no\ngeneral semantic formalism for representing meaning in terms of vectors. We\npresent a framework for natural language semantics in which words, phrases and\nsentences are all represented as vectors, based on a theoretical analysis which\nassumes that meaning is determined by context.\n In the theoretical analysis, we define a corpus model as a mathematical\nabstraction of a text corpus. The meaning of a string of words is assumed to be\na vector representing the contexts in which it occurs in the corpus model.\nBased on this assumption, we can show that the vector representations of words\ncan be considered as elements of an algebra over a field. We note that in\napplications of vector spaces to representing meanings of words there is an\nunderlying lattice structure; we interpret the partial ordering of the lattice\nas describing entailment between meanings. We also define the context-theoretic\nprobability of a string, and, based on this and the lattice structure, a degree\nof entailment between strings.\n We relate the framework to existing methods of composing vector-based\nrepresentations of meaning, and show that our approach generalises many of\nthese, including vector addition, component-wise multiplication, and the tensor\nproduct.\n