2019/09/10 by Kamrun Naher Keya, Keya, Kamrun Naher, Yannis Papanikolaou +3
Computer Science · Mathematics · Social Sciences · #Artificial intelligence #Coherence (philosophical gambling strategy) #Computation and Language (cs.CL) #Computational and Text Analysis Methods #Computer science #Embedding #FOS: Computer and information sciences #Generality #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine learning #Mathematics #Natural Language Processing Techniques #Natural language processing #Space (punctuation) #Topic Modeling #Topic model #Vector space #Word (group theory) #Word embedding #cs.CL #cs.IR #cs.LG
paper · pdf · doi:10.48550/arxiv.1909.04702
published in arXiv (Cornell University) (Cornell University)
arxiv created 2019/09/10 · arxiv updated 2019/09/12
Word embedding models such as the skip-gram learn vector representations of words' semantic relationships, and document embedding models learn similar representations for documents. On the other hand, topic models provide latent representations of the documents' topical themes. To get the benefits of these representations simultaneously, we propose a unifying algorithm, called neural embedding allocation (NEA), which deconstructs topic models into interpretable vector-space embeddings of words, topics, documents, authors, and so on, by learning neural embeddings to mimic the topic models. We showcase NEA's effectiveness and generality on LDA, author-topic models and the recently proposed mixed membership skip gram topic model and achieve better performance with the embeddings compared to several state-of-the-art models. Furthermore, we demonstrate that using NEA to smooth out the topics improves coherence scores over the original topic models when the number of topics is large.