2021/11/09 by S. Stanley Young, Young, S. Stanley, Warren B. Kindzierski +2
Agricultural and Biological Sciences · Mathematics · Medicine · #Applications (stat.AP) #Consumer Attitudes and Food Labeling #FOS: Computer and information sciences #Meat and Animal Product Quality #Nutritional Studies and Diet #stat.AP
paper · pdf · doi:10.48550/arxiv.2111.05337
20 pages, 1 figure, 7 Tables
arxiv created 2021/11/09 · openalex publication_date 2021/11/09 · arxiv updated 2021/11/11 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Background: Risk ratios or p-values from multiple, independent studies, observational or randomized, can be computationally combined to provide an overall assessment of a research question in meta-analysis. However, an irreproducibility crisis currently afflicts a wide range of scientific disciplines, including nutritional epidemiology. An evaluation was undertaken to assess the reliability of a meta-analysis examining the association between red and processed meat and selected human health effects (all-cause mortality, cardiovascular mortality, overall cancer mortality, breast cancer incidence, colorectal cancer incidence, type 2 diabetes incidence). Methods: The number of statistical tests and models were counted in 15 randomly selected base papers (14%) from 105 used in the meta-analysis. Relative risk with 95% confidence limits for 125 risk results were converted to p-values and p-value plots were constructed to evaluate the effect heterogeneity of the p-values. Results: The number of statistical tests possible in the 15 randomly selected base papers was large, median = 20,736 (interquartile range = 1,728 to 331,776). Each p-value plot for the six selected health effects showed either a random pattern (p-values > 0.05), or a two-component mixture with small p-values < 0.001 while other p-values appeared random. Given potentially large numbers of statistical tests conducted in the 15 selected base papers, questionable research practices cannot be ruled out as explanations for small p-values. Conclusions: This independent analysis, which complements the findings of the original meta-analysis, finds that the base papers used in the red and resulting processed meat meta-analysis do not provide evidence for the claimed health effects.