MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact\n Checking of Claims

2019/09/07 by Isabelle Augenstein, Augenstein, Isabelle, Christina Lioma +11 · 11 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #Expert finding and Q&A systems #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Misinformation and Its Impacts #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1909.03242

openalex publication_date 2019/09/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We contribute the largest publicly available dataset of naturally occurring\nfactual claims for the purpose of automatic claim verification. It is collected\nfrom 26 fact checking websites in English, paired with textual sources and rich\nmetadata, and labelled for veracity by human expert journalists. We present an\nin-depth analysis of the dataset, highlighting characteristics and challenges.\nFurther, we present results for automatic veracity prediction, both with\nestablished baselines and with a novel method for joint ranking of evidence\npages and predicting veracity that outperforms all baselines. Significant\nperformance increases are achieved by encoding evidence, and by modelling\nmetadata. Our best-performing model achieves a Macro F1 of 49.2%, showing that\nthis is a challenging testbed for claim veracity prediction.\n

Citations

• A simple but tough-to-beat baseline for the Fake News Challenge stance detection task
• The spread of true and false news online

Cited by

• MMM-Fact: A Multimodal, Multi-Domain Fact-Checking Dataset with Multi-Level Retrieval Difficulty
• A Comprehensive Dataset for Human vs. AI Generated Text Detection
• A Benchmark for Open-Domain Numerical Fact-Checking Enhanced by Claim Decomposition
• Bridging Language Gaps with Adaptive RAG: Improving Indonesian Language Question Answering
• Attacks by Content: Automated Fact-checking is an AI Security Issue
• Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning
• Think Right, Not More: Test-Time Scaling for Numerical Claim Verification
• Black-Box Hallucination Detection via Consistency Under the Uncertain Expression
• ShortCheck: Checkworthiness Detection of Multilingual Short-Form Videos
• Adversarial Attacks Against Automated Fact-Checking: A Survey
• TRUST-VL: An Explainable News Assistant for General Multimodal Misinformation Detection

Related

• MMM-Fact: A Multimodal, Multi-Domain Fact-Checking Dataset with Multi-Level Retrieval Difficulty
• FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information
• Fact Checking Beyond Training Set
• EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification
• Claim Extraction for Fact-Checking: Data, Models, and Automated Metrics