Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting

2023/07/04 by Chen Chen, Chen, Chen, Yufei Wang +7 · 6 citations
Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2307.01709

openalex publication_date 2023/07/04 · openalex created_date 2023/07/07 · openalex updated_date 2026/07/28

Abstract

Knowledge Graph Completion (KGC) often requires both KG structural and textual information to be effective. Pre-trained Language Models (PLMs) have been used to learn the textual information, usually under the fine-tune paradigm for the KGC task. However, the fine-tuned PLMs often overwhelmingly focus on the textual information and overlook structural knowledge. To tackle this issue, this paper proposes CSProm-KG (Conditional Soft Prompts for KGC) which maintains a balance between structural information and textual knowledge. CSProm-KG only tunes the parameters of Conditional Soft Prompts that are generated by the entities and relations representations. We verify the effectiveness of CSProm-KG on three popular static KGC benchmarks WN18RR, FB15K-237 and Wikidata5M, and two temporal KGC benchmarks ICEWS14 and ICEWS05-15. CSProm-KG outperforms competitive baseline models and sets new state-of-the-art on these benchmarks. We conduct further analysis to show (i) the effectiveness of our proposed components, (ii) the efficiency of CSProm-KG, and (iii) the flexibility of CSProm-KG.

Cited by

• Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
• From Knowledge to Treatment: Large Language Model Assisted Biomedical Concept Representation for Drug Repurposing
• Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning
• A Survey of Link Prediction in N-ary Knowledge Graphs
• KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Models
• Towards Structure-aware Model for Multi-modal Knowledge Graph Completion

Related

• KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs
• Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement
• KG-CF: Knowledge Graph Completion with Context Filtering under the Guidance of Large Language Models
• Making Large Language Models Perform Better in Knowledge Graph Completion
• Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?