2026/07/03 by Sam Sims, Jake Anders, Matthew Inglis +3 · 2 voices
Engineering · Social Sciences · #Design of experiments #Global Educational Reforms and Inequalities #Quasi-experiment #Random assignment #Research methodology #School Choice and Performance #Spatial Cognition and Navigation #Statistical analysis
paper · pdf · doi:10.1080/13803611.2026.2678611
published in Educational Research and Evaluation, 1-17 (Routledge)
openalex publication_date 2026/07/03 · openalex created_date 2026/07/04 · openalex updated_date 2026/07/22
Over the last twenty years, education researchers have increasingly conducted randomized experiments. Such experiments are usually designed in the hope of supporting comparisons of cost effectiveness across interventions, for example by using broad, standardized outcome measures. However, small effect sizes, wide confidence intervals, and heterogeneous treatment effects have limited their practical usefulness. We argue that, rather than directly informing decisions, randomized experiments should primarily play the role of testing theoretical models, which can in turn inform educators’ mental models. Since comparable effect sizes are not essential when testing theory, researchers can improve the power of such studies in various ways, including through the use of outcome measures more closely aligned with the theoretical constructs of interest. We also suggest that quasi-experimental methods may often be more suitable for informing education decisions, on the grounds that they can achieve greater precision at lower cost, with relatively moderate risks to internal validity.