Knowledge Mechanisms in Large Language Models: A Survey and Perspective

2024/07/22 by Mengru Wang, Wang, Mengru, Yunzhi Yao +23 · 23 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2407.15017

openalex publication_date 2024/07/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Understanding knowledge mechanisms in Large Language Models (LLMs) is crucial for advancing towards trustworthy AGI. This paper reviews knowledge mechanism analysis from a novel taxonomy including knowledge utilization and evolution. Knowledge utilization delves into the mechanism of memorization, comprehension and application, and creation. Knowledge evolution focuses on the dynamic progression of knowledge within individual and group LLMs. Moreover, we discuss what knowledge LLMs have learned, the reasons for the fragility of parametric knowledge, and the potential dark knowledge (hypothesis) that will be challenging to address. We hope this work can help understand knowledge in LLMs and provide insights for future research.

Cited by

• Some hypotheses on how chatbots work in problem-solving-driven conversations. Large Language Models as confirmation of the Innovation Illusion
• NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models
• Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detection
• Evontree: Ontology Rule-Guided Self-Evolution of Large Language Models
• Generalization or Memorization: Dynamic Decoding for Mode Steering
• DSCD: Large Language Model Detoxification with Self-Constrained Decoding
• STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Models
• Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
• DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search
• Provable Benefits of In-Tool Learning for Large Language Models
• Scaling Laws for Task-Stratified Knowledge in Post-Training Quantized Large Language Models
• Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models
• Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Models
• KScope: A Framework for Characterizing the Knowledge Status of Language Models
• TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Models
• One for All: Update Parameterized Knowledge Across Multiple Models
• COMPKE: Complex Question Answering under Knowledge Editing
• Benchmarking and Rethinking Knowledge Editing for Large Language Models
• Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms
• Concept Incongruence: An Exploration of Time and Death in Role Playing
• Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
• The Rise of Small Language Models in Healthcare: A Comprehensive Survey
• Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms

Related

• Knowledge Boundary of Large Language Models: A Survey
• Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms
• Large Language Models and Knowledge Graphs: Opportunities and Challenges
• Decoding Knowledge in Large Language Models: A Framework for Categorization and Comprehension
• Challenges and Contributing Factors in the Utilization of Large Language Models (LLMs)