Topic modeling

2006/01/01 by Hanna Wallach · 9 citations
Computer Science · Social Sciences · #Topic Modeling #Computational and Text Analysis Methods #Natural Language Processing Techniques

paper · doi:10.1145/1143844.1143967

openalex publication_date 2006/01/01 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/30

Abstract

Some models of textual corpora employ text generation methods involving n-gram statistics, while others use latent topic variables inferred using the "bag-of-words" assumption, in which word order is ignored. Previously, these methods have not been combined. In this work, I explore a hierarchical generative probabilistic model that incorporates both n-gram statistics and latent topic variables by extending a unigram topic model to include properties of a hierarchical Dirichlet bigram language model. The model hyperparameters are inferred using a Gibbs EM algorithm. On two data sets, each of 150 documents, the new model exhibits better predictive accuracy than either a hierarchical Dirichlet bigram language model or a unigram topic model. Additionally, the inferred topics are less dominated by function words than are topics discovered using unigram statistics, potentially making them more meaningful.

Cited by

• Evaluation methods for topic models
• Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks
• Mapping the unseen in practice: comparing latent Dirichlet allocation and BERTopic for navigating topic spaces
• Signal detection from spiked noise via asymmetrization
• Generating Synthetic Text Data to Evaluate Causal Inference Methods
• How Do Mothers and Fathers Talk About Parenting to Different Audiences?: Stereotypes and Audience Effects: An Analysis of r/Daddit, r/Mommit, and r/Parenting Using Topic Modelling
• Text Level Graph Neural Network for Text Classification
• Visualizing Topics with Multi-Word Expressions
• Hanna Wallach [wikipedia]