Accuracy of Effect Size Estimates from Published Psychological Research

2008/04/01 by Andrew Brand, M. T. Bradley, Lisa A. Best +1 · 2 citations
Decision Sciences · Psychology · Mathematics · Medicine · #Meta-analysis and systematic reviews #Psychometric Methodologies and Testing #scientometrics and bibliometrics research #Monte Carlo method #Statistical power #Sample size determination #Publication bias #Statistics #Publication #Meta-analysis #Econometrics #Computer science #Psychology #Mathematics #Medicine

paper · doi:10.2466/pms.106.2.645-649

openalex publication_date 2008/04/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/15

Abstract

A Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate that, when a predominant bias to publish studies with statistically significant results is coupled with inadequate statistical power, there will be an overestimation of effect sizes. The consequences such an effect size overestimation will then have on meta-analyses and power analyses are highlighted and discussed along with measures which can be taken to reduce the problem.

Citations

• The file drawer problem and tolerance for null results.
• Statistical methods in psychology journals: Guidelines and explanations.
• Things I have learned (so far).
• Finding the missing science: The fate of studies submitted for review by a human subjects committee.

Cited by

• Multiple Trials May Yield Exaggerated Effect Size Estimates
• Effect size [wikipedia]

Related

• Effect size estimates: Current use, calculations, and interpretation.
• Evaluating Effect Size in Psychological Research: Sense and Nonsense
• Standardized or simple effect size: What should be reported?
• Effect Size Estimation Fromt-Statistics in the Presence of Publication Bias
• Multiple Trials May Yield Exaggerated Effect Size Estimates