Interpreting Effect Sizes of Education Interventions
2020/04/27 by Matthew A. Kraft · 1 voice · 39 citations
Social Sciences · #School Choice and Performance
paper · doi:10.3102/0013189x20912798
openalex publication_date 2020/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/05/21
Abstract
Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.
Citations
Cited by
- The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions
- An Auditable Policy-Simulation Framework for Student Dropout in Intervention-Free Data
- Experimental education research: rethinking why, how, and when to use random assignment
- Cultural Relevance at Scale: The Effects of an Ethnic Studies Expansion on Academic Outcomes
- Preschool teachers provide fewer participation opportunities to working-class students than those from more privileged backgrounds
- Just the right amount of caution? Remote instruction and student performance in Sweden during the COVID-19 pandemic
- Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact
- Three-Sided Testing to Establish Practical Significance: A Tutorial
- (When) do teacher professional development interventions improve student Achievement? A meta-analysis of 128 high-quality studies
- Review of the impact on pupils of English academies
- Remote Text‐Supplemented Audiobook Intervention Supports Children's Explicit and Incidental Vocabulary Learning
- Failure to replicate effects of parent‐delivered early language intervention: Evidence from a randomised controlled trial with implications for universal language intervention
- Fundamental Mechanisms of Human Learning: Implications for Research and Practice in Education
- When Does Learning by Non-interactive Teaching Work? A Large-Scale Analysis of Learner Characteristics in a Classroom Setting
- Promotion of Elementary School Students' Health Literacy. [europepmc]
- What can be learned from growth mindset controversies? [europepmc]
- Teaching Metabolism in Upper-Division Undergraduate Biochemistry Courses using Online Computational Systems and Dynamical Models Improves Student Performance. [europepmc]
- Five Strategies for Optimizing Instructional Materials: Instructor- and Learner-Managed Cognitive Load. [europepmc]
- Video-feedback promotes sensitive limit-setting in parents of twin preschoolers: a randomized controlled trial. [europepmc]
- Effects of Montessori Education on the Academic, Cognitive, and Social Development of Disadvantaged Preschoolers: A Randomized Controlled Study in the French Public-School System. [europepmc]
- Effect of a Randomized Interventional School-Based Vision Program on Academic Performance of Students in Grades 3 to 7: A Cluster Randomized Clinical Trial. [europepmc]
- Teacher Mindsets Help Explain Where a Growth-Mindset Intervention Does and Doesn't Work. [europepmc]
- Implementing Remote Developmental Research: A Case Study of a Randomized Controlled Trial Language Intervention During COVID-19. [europepmc]
- Universal Mindfulness Training in Schools for Adolescents: a Scoping Review and Conceptual Model of Moderators, Mediators, and Implementation Factors. [europepmc]
- The policing paradox: Police stops predict youth's school disengagement via elevated psychological distress. [europepmc]
- Digital training for building resilience: Systematic review, meta-analysis, and meta-regression. [europepmc]
- Effectiveness and Implementation Outcome Measures of Mental Health Curriculum Intervention Using Social Media to Improve the Mental Health Literacy of Adolescents. [europepmc]
- Child's Play: Examining the Association Between Time Spent Playing and Child Mental Health. [europepmc]
- A synergistic mindsets intervention protects adolescents from stress. [europepmc]
- Online Racism and Mental Health Among Black American Adolescents in 2020. [europepmc]
- Schooling substantially improves intelligence, but neither lessens nor widens the impacts of socioeconomics and genetics. [europepmc]
- What are small, medium and large effect sizes for exercise treatments of tendinopathy? A systematic review and meta-analysis. [europepmc]
- Evaluation of a continuing professional development strategy on COVID-19 for 10 000 health workers in Ghana: a two-pronged approach. [europepmc]
- Policies and interventions to remove gender-related barriers to girls' school participation and learning in low- and middle-income countries: A systematic review of the evidence. [europepmc]
- A digital literacy model to narrow the digital literacy skills gap. [europepmc]
- Effects of the Jigsaw method on student educational outcomes: systematic review and meta-analyses. [europepmc]
- Why Meta-Analyses of Growth Mindset and Other Interventions Should Follow Best Practices for Examining Heterogeneity: Commentary on Macnamara and Burgoyne (2023) and Burnette et al. (2023). [europepmc]
- Power to Detect What? Considerations for Planning and Evaluating Sample Size. [europepmc]
- Promoting mental health in children and adolescents through digital technology: a systematic review and meta-analysis. [europepmc]
Discussions
Related