A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models

2025/05/25 by Utkarsh Sahu, Zhisheng Qi, Sahu, Utkarsh +14
Computer Science · #Advanced Graph Neural Networks #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Social and Information Networks (cs.SI) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.19286

openalex publication_date 2025/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models have been extensively studied as neural knowledge bases for their knowledge access, editability, reasoning, and explainability. However, few works focus on the structural patterns of their knowledge. Motivated by this gap, we investigate these structural patterns from a graph perspective. We quantify the knowledge of LLMs at both the triplet and entity levels, and analyze how it relates to graph structural properties such as node degree. Furthermore, we uncover the knowledge homophily, where topologically close entities exhibit similar levels of knowledgeability, which further motivates us to develop graph machine learning models to estimate entity knowledge based on its local neighbors. This model further enables valuable knowledge checking by selecting triplets less known to LLMs. Empirical results show that using selected triplets for fine-tuning leads to superior performance.

Citations

• Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
• Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base
• KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions
• STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases
• Can Language Models Act as Knowledge Bases at Scale?
• Distinguishing the Knowable from the Unknowable with Language Models
• KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphs
• Rethinking Language Models as Symbolic Knowledge Graphs
• Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?
• Language Models (Mostly) Know What They Know
• A Review on Language Models as Knowledge Bases
• Locating and Editing Factual Associations in GPT
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Tree Decomposed Graph Neural Network
• Is Homophily a Necessity for Graph Neural Networks?
• Knowledge Neurons in Pretrained Transformers
• Transformer Feed-Forward Layers Are Key-Value Memories
• Message Passing for Hyper-Relational Knowledge Graphs
• CoDEx: A Comprehensive Knowledge Graph Completion Benchmark
• Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
• How Much Knowledge Can You Pack Into the Parameters of a Language Model?
• Language Models as Knowledge Bases?
• Where Is the Semantic System? A Critical Review and Meta-Analysis of 120 Functional Neuroimaging Studies

Related

• Knowledge Homophily in Large Language Models
• Graph Machine Learning in the Era of Large Language Models (LLMs)
• GUNDAM: Aligning Large Language Models with Graph Understanding
• Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models
• GraphGhost: Tracing Structures Behind Large Language Models