2019/10/15 by Ben Weidmann, Weidmann, Ben, Luke Miratrix +1
Decision Sciences · Mathematics · Social Sciences · #Advanced Causal Inference Techniques #Applications (stat.AP) #Educational Assessment and Improvement #FOS: Computer and information sciences #School Choice and Performance
paper · pdf · doi:10.48550/arxiv.1910.07091
openalex publication_date 2019/10/15 · openalex created_date 2019/10/25 · openalex updated_date 2026/07/28
This study examines whether unobserved factors substantially bias education evaluations that rely on the Conditional Independence Assumption. We add 14 new within-study comparisons to the literature, all from primary schools in England. Across these 14 studies, we generate 42 estimates of selection bias using a simple matching approach. A meta-analysis of the estimates suggests that the distribution of underlying bias is centered around zero. The mean absolute value of estimated bias is 0.03σ, and none of the 42 estimates are larger than 0.11σ. Results are similar for math, reading and writing outcomes. Overall, we find no evidence of substantial selection bias due to unobserved characteristics. These findings may not generalise easily to other settings or to more radical educational interventions, but they do suggest that non-experimental approaches could play a greater role than they currently do in generating reliable causal evidence for school education.