Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
2023/05/22 by Jian Xie, Xie Jian, Kai Zhang +8 · 1 voice · 87 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL
paper · pdf · doi:10.48550/arxiv.2305.13300
arxiv published 2023/05/22 · arxiv updated 2024/02/27
Abstract
By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory. However, how receptive are LLMs to such external evidence, especially when the evidence conflicts with their parametric memory? We present the first comprehensive and controlled investigation into the behavior of LLMs when encountering knowledge conflicts. We propose a systematic framework to elicit high-quality parametric memory from LLMs and construct the corresponding counter-memory, which enables us to conduct a series of controlled experiments. Our investigation reveals seemingly contradicting behaviors of LLMs. On the one hand, different from prior wisdom, we find that LLMs can be highly receptive to external evidence even when that conflicts with their parametric memory, given that the external evidence is coherent and convincing. On the other hand, LLMs also demonstrate a strong confirmation bias when the external evidence contains some information that is consistent with their parametric memory, despite being presented with conflicting evidence at the same time. These results pose important implications that are worth careful consideration for the further development and deployment of tool- and retrieval-augmented LLMs. Resources are available at https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.
Cited by
- TokenMem: Faithful Knowledge Injection for Frozen LLMs
- A Unified Definition of Hallucination: It's The World Model, Stupid!
- ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions
- A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledge
- The Illusion of Certainty: Uncertainty Quantification for LLMs Fails under Ambiguity
- Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models
- Mitigating Modal Imbalance in Multimodal Reasoning
- Prior Knowledge Makes It Possible: From Sublinear Graph Algorithms to LLM Test-Time Methods
- How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs
- Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
- A Video Is Not Worth a Thousand Words
- NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledge
- When Facts Change: Probing LLMs on Evolving Knowledge with evolveQA
- That's Deprecated! Understanding, Detecting, and Steering Knowledge Conflicts in Language Models for Code Generation
- LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval
- Probing Latent Knowledge Conflict for Faithful Retrieval-Augmented Generation
- When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation
- Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
- BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
- Training Dynamics of Parametric and In-Context Knowledge Utilization in Language Models
- Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Models
- LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions
- How Persuasive is Your Context?
- REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Prompting
- Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts
- Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions
- Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution
- Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Models
- Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
- Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering
- LingVarBench: Benchmarking LLM for Automated Named Entity Recognition in Structured Synthetic Spoken Transcriptions
- Format as a Prior: Quantifying and Analyzing Bias in LLMs for Heterogeneous Data
- Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinking
- KCR: Resolving Long-Context Knowledge Conflicts via Reasoning in LLMs
- MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generation
- Your AI, Not Your View: The Bias of LLMs in Investment Analysis
- Exploring the Impact of Instruction-Tuning on LLM's Susceptibility to Misinformation
- Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
- FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation
- Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
- Robust Multimodal Large Language Models Against Modality Conflict
- Steering Information Utility in Key-Value Memory for Language Model Post-Training
- Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
- Small Encoders Can Rival Large Decoders in Detecting Groundedness
- KScope: A Framework for Characterizing the Knowledge Status of Language Models
- Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs
- Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
- The Compositional Architecture of Regret in Large Language Models
- Question Answering under Temporal Conflict: Evaluating and Organizing Evolving Knowledge with LLMs
- Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict
- Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
- When to Trust Context: Self-Reflective Debates for Context Reliability
- Micro-Act: Mitigating Knowledge Conflict in LLM-based RAG via Actionable Self-Reasoning
- Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement
- SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models
- CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention
- ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation
- If Pigs Could Fly... Can LLMs Logically Reason Through Counterfactuals?
- How does Misinformation Affect Large Language Model Behaviors and Preferences?
- Evaluating and Steering Modality Preferences in Multimodal Large Language Model
- Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models
- CUB: Benchmarking Context Utilisation Techniques for Language Models
- Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning
- Pre-training Limited Memory Language Models with Internal and External Knowledge
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation
- UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models
- SAFE: Improving LLM Systems using Sentence-Level In-generation Attribution
- GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents
- Towards Contamination Resistant Benchmarks
- Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
- Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models
- DenialRAG: Single-Document RAG Poisoning via Embedded Parametric Denial
- Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models
- ConSens: Assessing context grounding in open-book question answering
- RDF-Based Structured Quality Assessment Representation of Multilingual LLM Evaluations
- Safety and accuracy follow different scaling laws in clinical large language models
- Can LLMs Introspect? A Reality Check
- NanoKnow: How to Know What Your Language Model Knows
- Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence
- Conflicts in Texts: Data, Implications and Challenges
- CORG: Generating Answers from Complex, Interrelated Contexts
- Mechanisms of Prompt-Induced Hallucination in Vision-Language Models
- Whose Facts Win? LLM Source Preferences under Knowledge Conflicts
- HalluLens: LLM Hallucination Benchmark
- PURPOSE: Poisoning Conflict Resolution in RAG via Proxy-Fact-Grounded Updates
- MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluation
- Exploiting Contextual Knowledge in LLMs through V-usable Information based Layer Enhancement
- Harnessing the Unseen: The Hidden Influence of Intrinsic Knowledge in Long-Context Language Models
- Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling
Discussions
Related