2021/12/08 by Ganqiang Ye, Ye, Ganqiang, Wen Zhang +8 · 1 citation
Computer Science · Decision Sciences · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Data Quality and Management #FOS: Computer and information sciences #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2112.04087
openalex publication_date 2021/12/08 · openalex created_date 2021/12/31 · openalex updated_date 2026/07/28
Representation learning models for Knowledge Graphs (KG) have proven to be effective in encoding structural information and performing reasoning over KGs. In this paper, we propose a novel pre-training-then-fine-tuning framework for knowledge graph representation learning, in which a KG model is firstly pre-trained with triple classification task, followed by discriminative fine-tuning on specific downstream tasks such as entity type prediction and entity alignment. Drawing on the general ideas of learning deep contextualized word representations in typical pre-trained language models, we propose SCoP to learn pre-trained KG representations with structural and contextual triples of the target triple encoded. Experimental results demonstrate that fine-tuning SCoP not only outperforms results of baselines on a portfolio of downstream tasks but also avoids tedious task-specific model design and parameter training.