Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method

2023/09/30 by Xuan Zhang, Wei Gao, Zhang, Xuan +1 · 17 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Misinformation and Its Impacts #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.00305

openalex publication_date 2023/09/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

While large pre-trained language models (LLMs) have shown their impressive capabilities in various NLP tasks, they are still under-explored in the misinformation domain. In this paper, we examine LLMs with in-context learning (ICL) for news claim verification, and find that only with 4-shot demonstration examples, the performance of several prompting methods can be comparable with previous supervised models. To further boost performance, we introduce a Hierarchical Step-by-Step (HiSS) prompting method which directs LLMs to separate a claim into several subclaims and then verify each of them via multiple questions-answering steps progressively. Experiment results on two public misinformation datasets show that HiSS prompting outperforms state-of-the-art fully-supervised approach and strong few-shot ICL-enabled baselines.

Cited by

• Verification-Notebook Learning for Source-Aware Multimodal Misinformation Detection
• Explainable Ethical Assessment on Human Behaviors by Generating Conflicting Social Norms
• Towards Unification of Hallucination Detection and Fact Verification for Large Language Models
• REFLEX: Self-Refining Explainable Fact-Checking via Disentangling Truth into Style and Substance
• Insight-A: Attribution-aware for Multimodal Misinformation Detection
• When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection
• ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction
• Retrieval and Argumentation Enhanced Multi-Agent LLMs for Judgmental Forecasting
• MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs
• Efficiently Executing High-throughput Lightweight LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
• Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
• Towards Real-Time Fake News Detection under Evidence Scarcity
• FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs
• Automatic Fact-checking in English and Telugu
• Scaling Up Throughput-oriented LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
• Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
• The Missing Parts: Augmenting Fact Verification with Half-Truth Detection

Related

• Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs
• GPTs at Factify 2022: Prompt Aided Fact-Verification
• Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
• Fact-checking AI-generated news reports: Can LLMs catch their own lies?
• RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking