vix.ing · top · new · best · stats · spec

Annotation Tool and Dataset for Fact-Checking Podcasts

2025/02/03 by Vinay Setty, Setty, Vinay, Adam James Becker +1 · 2 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Peer-to-Peer Network Technologies #Radio, Podcasts, and Digital Media #Web Data Mining and Analysis

paper · pdf · doi:10.48550/arxiv.2502.01402

openalex publication_date 2025/02/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Podcasts are a popular medium on the web, featuring diverse and multilingual content that often includes unverified claims. Fact-checking podcasts is a challenging task, requiring transcription, annotation, and claim verification, all while preserving the contextual details of spoken content. Our tool offers a novel approach to tackle these challenges by enabling real-time annotation of podcasts during playback. This unique capability allows users to listen to the podcast and annotate key elements, such as check-worthy claims, claim spans, and contextual errors, simultaneously. By integrating advanced transcription models like OpenAI's Whisper and leveraging crowdsourced annotations, we create high-quality datasets to fine-tune multilingual transformer models such as XLM-RoBERTa for tasks like claim detection and stance classification. Furthermore, we release the annotated podcast transcripts and sample annotations with preliminary experiments.

Cited by

Related