2009/07/15 by Andrew Gelman, Jennifer Hill, Gelman, Andrew +3 · 2 voices · 17 citations
Decision Sciences · Mathematics · Psychology · #Artificial intelligence #Bayesian probability #Computer science #Data mining #Econometrics #Hierarchical database model #Mathematics #Multilevel model #Multiple comparisons problem #Optimal Experimental Design Methods #Perspective (graphical) #Pooling #Psychology #Statistical Methods and Bayesian Inference #Statistical Methods in Clinical Trials #Statistics #Type I and type II errors #Worry #stat.AP #stat.ME
paper · pdf · doi:10.48550/arxiv.0907.2478
published in arXiv (Cornell University) (Cornell University)
arxiv created 2009/07/15 · openalex publication_date 2009/07/15 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p-values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.