KG-BERT: BERT for Knowledge Graph Completion

2019/09/07 by Liang Yao, Yao, Liang, Chengsheng Mao +3 · 34 citations
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Semantic Web and Ontologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1909.03193

openalex publication_date 2019/09/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Knowledge graphs are important resources for many artificial intelligence tasks but often suffer from incompleteness. In this work, we propose to use pre-trained language models for knowledge graph completion. We treat triples in knowledge graphs as textual sequences and propose a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples. Our method takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model. Experimental results on multiple benchmark knowledge graphs show that our method can achieve state-of-the-art performance in triple classification, link prediction and relation prediction tasks.

Citations

• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex\n Space

Cited by

• GR-Agent: Adaptive Graph Reasoning Agent under Incomplete Knowledge
• Semantic-Constrained Federated Aggregation: Convergence Theory and Privacy-Utility Bounds for Knowledge-Enhanced Distributed Learning
• Decoding Student Minds: Leveraging Conversational Agents for Psychological and Learning Analysis
• VisKnow: Constructing Visual Knowledge Base for Object Understanding
• Wikontic: Constructing Wikidata-Aligned, Ontology-Aware Knowledge Graphs with Large Language Models
• Beyond Generation: Multi-Hop Reasoning for Factual Accuracy in Vision-Language Models
• OpenGloss: A Synthetic Encyclopedic Dictionary and Semantic Knowledge Graph
• Diagnosing and Mitigating Semantic Inconsistencies in Wikidata's Classification Hierarchy
• Hybrid Fact-Checking that Integrates Knowledge Graphs, Large Language Models, and Search-Based Retrieval Agents Improves Interpretable Claim Verification
• Toward a Unified Statistical Theory of Unsupervised Pretraining and Supervised Neural Knowledge Graph Learning
• Uncovering Gaps Between RFC Updates and TCP/IP Implementations: LLM-Facilitated Differential Checks on Intermediate Representations
• LightKGG: Simple and Efficient Knowledge Graph Generation from Textual Data
• Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
• ELMM: Efficient Lightweight Multimodal Large Language Models for Multimodal Knowledge Graph Completion
• ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAG
• From Knowledge to Treatment: Large Language Model Assisted Biomedical Concept Representation for Drug Repurposing
• Semantic-Condition Tuning: Fusing Graph Context with Large Language Models for Knowledge Graph Completion
• CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning
• Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning
• Type-Less yet Type-Aware Inductive Link Prediction with Pretrained Language Models
• Towards Structured Knowledge: Advancing Triple Extraction from Regional Trade Agreements using Large Language Models
• Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching
• Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
• Towards Explainable Job Title Matching: Leveraging Semantic Textual Relatedness and Knowledge Graphs
• KLIPA: A Knowledge Graph and LLM-Driven QA Framework for IP Analysis
• Automated Hierarchical Graph Construction for Multi-source Electronic Health Records
• SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion
• Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning
• What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledge
• CEAR: Cross-Entity Aware Reranker for Knowledge Base Completion
• Guided Navigation in Knowledge-Dense Environments: Structured Semantic Exploration with Guidance Graphs
• Ontology-Enhanced Knowledge Graph Completion using Large Language Models
• Commonsense Knowledge Mining from Term Definitions
• Knowledge Graph Refinement based on Triplet BERT-Networks

Related

• Exploring Large Language Models for Knowledge Graph Completion
• Knowledge Graph Refinement based on Triplet BERT-Networks
• Bridge: A Unified Framework to Knowledge Graph Completion via Language Models and Knowledge Representation
• GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion
• Pre-training Transformers for Knowledge Graph Completion