Fact-Checking with Contextual Narratives: Leveraging Retrieval-Augmented LLMs for Social Media Analysis

2025/04/14 by Dey, Arka Ujjal, Awan, Muhammad Junaid, Channing, Georgia +2 · 1 citation
#FOS: Computer and information sciences #Multimedia (cs.MM)

paper · doi:10.48550/arxiv.2504.10166

Abstract

We propose CRAVE (Cluster-based Retrieval Augmented Verification with Explanation); a novel framework that integrates retrieval-augmented Large Language Models (LLMs) with clustering techniques to address fact-checking challenges on social media. CRAVE automatically retrieves multimodal evidence from diverse, often contradictory, sources. Evidence is clustered into coherent narratives, and evaluated via an LLM-based judge to deliver fact-checking verdicts explained by evidence summaries. By synthesizing evidence from both text and image modalities and incorporating agent-based refinement, CRAVE ensures consistency and diversity in evidence representation. Comprehensive experiments demonstrate CRAVE's efficacy in retrieval precision, clustering quality, and judgment accuracy, showcasing its potential as a robust decision-support tool for fact-checkers.

Citations

• TrumorGPT: Graph-Based Retrieval-Augmented Large Language Model for Fact-Checking
• GraphCheck: Multipath Fact-Checking with Entity-Relationship Graphs
• COVE: COntext and VEracity prediction for out-of-context images
• DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts
• Supernotes: Driving Consensus in Crowd-Sourced Fact-Checking
• "Image, Tell me your story!" Predicting the original meta-context of visual misinformation
• The Llama 3 Herd of Models
• Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?
• MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs
• SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection
• Debating with More Persuasive LLMs Leads to More Truthful Answers
• RED-DOT: Multimodal Fact-checking via Relevant Evidence Detection
• Improved Baselines with Visual Instruction Tuning
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia
• InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
• VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias
• Detecting and Grounding Multi-Modal Media Manipulation
• Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
• NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• The spread of true and false news online
• Posttruth, Truthiness, and Alternative Facts: Information Behavior and Critical Information Consumption for a New Age
• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
• COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learning
• Social Media and Fake News in the 2016 Election

Cited by

• SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Services

Related

• Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
• Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking
• Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
• The Perils & Promises of Fact-checking with Large Language Models
• Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models