WithdrarXiv: A Large-Scale Dataset for Retraction Study
2024/12/04 by Delip Rao, Rao, Delip, Jonathan Young +5 · 9 voices · 4 citations
#cs.CL #cs.DL #cs.LG
paper · pdf · doi:10.48550/arxiv.2412.03775
Abstract
Retractions play a vital role in maintaining scientific integrity, yet systematic studies of retractions in computer science and other STEM fields remain scarce. We present WithdrarXiv, the first large-scale dataset of withdrawn papers from arXiv, containing over 14,000 papers and their associated retraction comments spanning the repository's entire history through September 2024. Through careful analysis of author comments, we develop a comprehensive taxonomy of retraction reasons, identifying 10 distinct categories ranging from critical errors to policy violations. We demonstrate a simple yet highly accurate zero-shot automatic categorization of retraction reasons, achieving a weighted average F1-score of 0.96. Additionally, we release WithdrarXiv-SciFy, an enriched version including scripts for parsed full-text PDFs, specifically designed to enable research in scientific feasibility studies, claim verification, and automated theorem proving. These findings provide valuable insights for improving scientific quality control and automated verification systems. Finally, and most importantly, we discuss ethical issues and take a number of steps to implement responsible data release while fostering open science in this area.
Cited by
Discussions
- 😳 WithdrarXiv 🙏 - Dataset of 14K+ withdrawn arXiv papers - associated retraction comments - entire history through 09/24 - taxonomy of retraction reasons, from critical errors to policy violations [bsky, 158 points, 5 comments]
- A new large-scale dataset of more than 14k withdrawn/retracted preprints from arXiv from @deliprao.bsky.social and co. arxiv.org/abs/2412.03775 #SciSci #MetaScience [bsky, 20 points, 0 comments]
- "WithdrarXiv: A Large-Scale Dataset for Retraction Study" New contender for most ridiculous Xiv usage (despite being a cool sounding project). arxiv.org/abs/2412.03775 [bsky, 4 points, 0 comments]
- WithdrarXiv: A Large-Scale Dataset for Retraction Study [hn, 3 points, 0 comments]
- "the first large-scale dataset of withdrawn papers from arXiv, containing over 14,000 papers and their associated retraction comments spanning the repository's entire history through September 2024." [bsky, 2 points, 0 comments]
- WithdrarXiv, the first large-scale dataset of withdrawn papers from arXiv, containing over 14,000 papers and their associated retraction comments spanning the repository's entire history through Septe [bsky, 2 points, 0 comments]
- > the first large-scale dataset of withdrawn papers from arXiv, containing over 14,000 papers and their associated retraction comments spanning the repository's entire history through September 2024 a [bsky, 1 points, 0 comments]
- The paper itself: arxiv.org/abs/2412.03775 [bsky, 0 points, 0 comments]
- Aside from its witty name, WithdrarXiv gives a pretty interesting insight: "Unlike biomedical fields where plagiarism often leads withdrawals, most arXiv retractions stem from factual or methodologica [bsky, 0 points, 0 comments]
Related