2015/03/13 by Megan L. Head, Luke Holman, Robert Lanfear +2 · 2 citations
Decision Sciences · Mathematics · #scientometrics and bibliometrics research #Meta-analysis and systematic reviews #Scientific Computing and Data Management #Hacker #Biology #Publication bias #Statistical hypothesis testing #Data science #Statistics #MEDLINE #Computer science #Mathematics #Computer security
paper · pdf · doi:10.1371/journal.pbio.1002106
openalex publication_date 2015/03/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as "p-hacking," occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.