How LLMs Fail to Support Fact-Checking

2025/02/28 by Proma, Adiba Mahbub, Pate, Neeley, Druckman, James +3 · 1 citation
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2503.01902

Abstract

While Large Language Models (LLMs) can amplify online misinformation, they also show promise in tackling misinformation. In this paper, we empirically study the capabilities of three LLMs -- ChatGPT, Gemini, and Claude -- in countering political misinformation. We implement a two-step, chain-of-thought prompting approach, where models first identify credible sources for a given claim and then generate persuasive responses. Our findings suggest that models struggle to ground their responses in real news sources, and tend to prefer citing left-leaning sources. We also observe varying degrees of response diversity among models. Our findings highlight concerns about using LLMs for fact-checking through only prompt-engineering, emphasizing the need for more robust guardrails. Our results have implications for both researchers and non-technical users.

Cited by

• Epistemic Fragility in Large Language Models: Prompt Framing Systematically Modulates Misinformation Correction

Related

• On the Effectiveness of Fact Checking Information from Politically Congruent and Incongruent Large Language Models
• Fact-checking with Generative AI: A Systematic Cross-Topic Examination of LLMs Capacity to Detect Veracity of Political Information
• Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search
• Combating Misinformation in the Age of LLMs: Opportunities and Challenges
• Evaluating open-source Large Language Models for automated fact-checking