‘Replication games’ test the robustness of social-science studies

2026/04/01 · 1 voice
Decision Sciences · #Scientific Computing and Data Management #Meta-analysis and systematic reviews #scientometrics and bibliometrics research

paper · doi:10.1038/d41586-026-00684-9

Abstract

A large-scale assessment of the reproducibility and robustness of 110 articles published in leading economics and political-science journals finds that more than 85% were computationally reproducible, and 72% of statistically significant results remained significant and in the same direction in robustness checks. About 25% of the evaluated articles contained non‑trivial coding errors. Work to replicate studies in economics and political science highlights the importance of sharing data and code.

Citations

• Estimating the reproducibility of psychological science
• Transparency, Reproducibility, and the Credibility of Economics Research

Discussions

• 11/ Research briefing here: doi.org/10.1038/d415... And a few other articles on reproducibility worth checking: Link to April 2nd issue: www.nature.com/nature/volum... [bsky, 10 points, 0 comments]

Related

• Computational Reproducibility in Computational Social Science
• Collusive and adversarial replication
• Replication code availability over time and across fields: Evidence from the German Socio‐Economic Panel
• Code sharing and reproducibility in survey-based social research: evidence from a large-scale audit
• The reliability of replications: a study in computational reproductions