Enriching BERT with Knowledge Graph Embeddings for Document Classification

2019/09/18 by Malte Ostendorff, Ostendorff, Malte, Peter Bourgonje +11 · 1 voice · 1 citation
Computer Science · #Authorship Attribution and Profiling #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.CL #cs.IR #cs.LG

paper · pdf · doi:10.48550/arxiv.1909.08402

openalex publication_date 2019/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we focus on the classification of books using short descriptive texts (cover blurbs) and additional metadata. Building upon BERT, a deep neural language model, we demonstrate how to combine text representations with metadata and knowledge graph embeddings, which encode author information. Compared to the standard BERT approach we achieve considerably better results for the classification task. For a more coarse-grained classification using eight labels we achieve an F1- score of 87.20, while a detailed classification using 343 labels yields an F1-score of 64.70. We make the source code and trained models of our experiments publicly available

Cited by

• FuDoBa: Fusing Document and Knowledge Graph-based Representations with Bayesian Optimisation

Discussions

• Enriching Bert with Knowledge Graph Embeddings for Document Classification [hn, 9 points, 0 comments]

Related

• Text classification problems via BERT embedding method and graph\n convolutional neural network
• K-BERT: Enabling Language Representation with Knowledge Graph
• DocBERT: BERT for Document Classification
• Layer-wise Guided Training for BERT: Learning Incrementally Refined\n Document Representations
• A Sentence-level Hierarchical BERT Model for Document Classification with Limited Labelled Data