Neural Topic Modeling by Incorporating Document Relationship Graph

2020/09/29 by Deyu Zhou, Zhou, Deyu, Xuemeng Hu +3 · 3 citations
Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2009.13972

openalex publication_date 2020/09/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Graph Neural Networks (GNNs) that capture the relationships between graph nodes via message passing have been a hot research direction in the natural language processing community. In this paper, we propose Graph Topic Model (GTM), a GNN based neural topic model that represents a corpus as a document relationship graph. Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences. By introducing the graph structure, the relationships between documents are established through their shared words and thus the topical representation of a document is enriched by aggregating information from its neighboring nodes using graph convolution. Extensive experiments on three datasets were conducted and the results demonstrate the effectiveness of the proposed approach.

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Probabilistic topic models
• Gated Graph Sequence Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Wasserstein Auto-Encoders
• Graph Neural Networks: A Review of Methods and Applications

Cited by

• Importance-aware Topic Modeling for Discovering Public Transit Risk from Noisy Social Media
• Unsupervised Multimodal Graph-based Model for Geo-social Analysis
• HAMLET: Healthcare-focused Adaptive Multilingual Learning Embedding-based Topic Modeling

Related

• Neural Graph Embedding Methods for Natural Language Processing
• Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs
• Improving Graph-Based Text Representations with Character and Word Level N-grams
• Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks
• Graph-based Neural Multi-Document Summarization