Unsupervised Question Answering for Fact-Checking

2019/10/16 by Mayank Jobanputra, Jobanputra, Mayank
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1910.07154

FEVER - 19 (EMNLP)

arxiv created 2019/10/16 · openalex publication_date 2019/10/16 · arxiv updated 2019/10/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent Deep Learning (DL) models have succeeded in achieving human-level accuracy on various natural language tasks such as question-answering, natural language inference (NLI), and textual entailment. These tasks not only require the contextual knowledge but also the reasoning abilities to be solved efficiently. In this paper, we propose an unsupervised question-answering based approach for a similar task, fact-checking. We transform the FEVER dataset into a Cloze-task by masking named entities provided in the claims. To predict the answer token, we utilize pre-trained Bidirectional Encoder Representations from Transformers (BERT). The classifier computes label based on the correctly answered questions and a threshold. Currently, the classifier is able to classify the claims as "SUPPORTS" and "MANUAL_REVIEW". This approach achieves a label accuracy of 80.2% on the development set and 80.25% on the test set of the transformed dataset.

Citations

• Learning to Ask: Neural Question Generation for Reading Comprehension
• Automated Fact Checking: Task formulations, methods and future directions
• UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification

Related

• Language Models as Fact Checkers?
• Fully Automated Fact Checking Using External Sources
• FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering
• QA-NatVer: Question Answering for Natural Logic-based Fact Verification
• Applying Deep Learning to Answer Selection: A Study and An Open Task