2025/12/12 by Désirée Nießen, Caroline Poppa, Jessica Daikeler +3 · 1 voice
Decision Sciences · Mathematics · #Advanced Causal Inference Techniques #Meta-analysis and systematic reviews #scientometrics and bibliometrics research
paper · doi:10.31234/osf.io/phk3a_v1
openalex created_date 2025/12/12 · openalex publication_date 2025/12/12 · openalex updated_date 2026/07/14
Science has long struggled with unsuccessful replications and reproductions, a challenge that raises questions about the robustness and credibility of research findings. One potential contributing factor is selective reporting or nonpublication of certain results based on their direction or strength, leading to publication bias. To better understand its extent and risk factors, we examined 178 successful study submissions to two German probabilistic panels (GESIS Panel, SOEP-IS), fielded between 2013 and 2021, and compared the hypotheses and exploratory analyses stated in the original study submissions with their presentation in subsequent publications. Using a coding scheme, we extracted information on the submissions, publications, and authors, and investigated which characteristics are associated with publication bias. Experimental study design, third-party funding, preregistration of hypotheses, and economic studies were the strongest and most stable predictors of lower risk of publication bias across multiple outcomes: publication likelihood, test and hit rate of hypotheses. The study highlights the complexity and multi-faceted nature of publication bias in the social, economic, and behavioral sciences, and shows that researchers (e.g., preregistration), funding agencies (e.g., incentives), and journals (e.g., peer review) play distinct roles in shaping publication bias. It thereby points to areas for reform in the research and publication process.