2019/11/25 by Silèye Ba, Sileye 0. Ba, Ba, Sileye 0.
Computer Science · Mathematics · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1911.10924
10 pages, 7 figures, 3 tables
arxiv created 2019/11/25 · openalex publication_date 2019/11/25 · arxiv updated 2019/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper we present a model for unsupervised topic discovery in texts corpora. The proposed model uses documents, words, and topics lookup table embedding as neural network model parameters to build probabilities of words given topics, and probabilities of topics given documents. These probabilities are used to recover by marginalization probabilities of words given documents. For very large corpora where the number of documents can be in the order of billions, using a neural auto-encoder based document embedding is more scalable then using a lookup table embedding as classically done. We thus extended the lookup based document embedding model to continuous auto-encoder based model. Our models are trained using probabilistic latent semantic analysis (PLSA) assumptions. We evaluated our models on six datasets with a rich variety of contents. Conducted experiments demonstrate that the proposed neural topic models are very effective in capturing relevant topics. Furthermore, considering perplexity metric, conducted evaluation benchmarks show that our topic models outperform latent Dirichlet allocation (LDA) model which is classically used to address topic discovery tasks.