2020/07/05 by Aaron Fisher, Aaron J. Fisher, Fisher, Aaron
Mathematics · #Advanced Causal Inference Techniques #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2007.02514
openalex publication_date 2020/07/05 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
Popular guidance on observational data analysis states that outcomes should\nbe blinded when determining matching criteria or propensity scores. Such a\nblinding is informally said to maintain the "objectivity" of the analysis, and\nto prevent analysts from fishing for positive results by exploiting chance\nimbalances. Contrary to this notion, we show that outcome blinding is not a\nsufficient safeguard against fishing. Blinded and unblinded analysts can\nproduce bias of the same order of magnitude in cases where the outcomes can be\napproximately predicted from baseline covariates. We illustrate this\nvulnerability with a combination of analytical results and simulations.\nFinally, to show that outcome blinding is not necessary to prevent bias, we\noutline an alternative sample partitioning procedure for estimating the average\ntreatment effect on the controls, or the average treatment effect on the\ntreated. This procedure uses all of the the outcome data from all partitions in\nthe final analysis step, but does not require the analysis to not be fully\nprespecified.\n