PPKE: Knowledge Representation Learning by Path-based Pre-training

2020/12/07 by He, Bin, Zhou, Di, Xie, Jing +3
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2012.03573

Abstract

Entities may have complex interactions in a knowledge graph (KG), such as multi-step relationships, which can be viewed as graph contextual information of the entities. Traditional knowledge representation learning (KRL) methods usually treat a single triple as a training unit, and neglect most of the graph contextual information exists in the topological structure of KGs. In this study, we propose a Path-based Pre-training model to learn Knowledge Embeddings, called PPKE, which aims to integrate more graph contextual information between entities into the KRL model. Experiments demonstrate that our model achieves state-of-the-art results on several benchmark datasets for link prediction and relation prediction tasks, indicating that our model provides a feasible way to take advantage of graph contextual information in KGs.

Related

• Improving Knowledge Graph Representation Learning by Structure Contextual Pre-training
• KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction
• Integrating Graph Contextualized Knowledge into Pre-trained Language Models
• Rule-Guided Joint Embedding Learning over Knowledge Graphs
• Walk-and-Relate: A Random-Walk-based Algorithm for Representation Learning on Sparse Knowledge Graphs