2021/12/15 by Mariusz Maziarz · 1 citation
Decision Sciences · Mathematics · Economics, Econometrics and Finance · Psychology · Medicine · #Meta-analysis and systematic reviews #Statistical Methods in Clinical Trials #Health Systems, Economic Evaluations, Quality of Life #Meta-analysis #Scope (computer science) #Systematic review #Argument (complex analysis) #Publication bias #Psychological intervention #MEDLINE #Confidence interval #Psychology #Computer science #Econometrics #Cognitive psychology #Medicine #Statistics #Mathematics #Psychiatry
paper · pdf · doi:10.1016/j.shpsa.2021.11.007
openalex publication_date 2021/12/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04
Literature-based meta-analysis is a standard technique applied to pool results of individual studies used in medicine and social sciences. It has been criticized for being too malleable to constrain results, averaging incomparable values, lacking a measure of evidence's strength, and problems with a systematic bias of individual studies. We argue against using literature-based meta-analysis of RCTs to assess treatment efficacy and show that therapeutic decisions based on meta-analytic average are not optimal given the full scope of existing evidence. The argument proceeds with discussing examples and analyzing the properties of some standard meta-analytic techniques. First, we demonstrate that meta-analysis can lead to reporting statistically significant results despite the treatment's limited efficacy. Second, we show that meta-analytic confidence intervals are too narrow compared to the variability of treatment outcomes reported by individual studies. Third, we argue that literature-based meta-analysis is not a reliable measurement instrument. Finally, we show that meta-analysis averages out the differences among studies and leads to a loss of information. Despite these problems, literature-based meta-analysis is useful for the assessment of harms. We support two alternative approaches to evidence amalgamation: meta-analysis of individual patient data (IPD) and qualitative review employing mechanistic evidence.