Effect Sizes in Cluster-Randomized Designs

2007/08/10 by Larry V. Hedges · 1 voice · 11 citations
Agricultural and Biological Sciences · Environmental Science · Psychology · #Behavioral and Psychological Studies #Diverse Educational Innovations Studies #Urban Green Space and Health

paper · doi:10.3102/1076998606298043

openalex publication_date 2007/08/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/16

Abstract

Multisite research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies (and corresponding quasi-experiments) with other studies and to combine information across studies in meta-analyses. This article addresses the problem of defining effect sizes in multilevel designs and computing estimates of those effect sizes and their standard errors from information that is likely to be reported in journal articles. Three effect sizes are defined corresponding to different standardizations. Estimators of each effect size index are also presented along with their sampling distributions (including standard errors).

Cited by

• A functional trait perspective on restored temperate grassland responses to changing winter insulation and managed disturbance by fire
• Using Extant Data to Improve Estimation of the Standardized Mean Difference
• Does spelling instruction make students better spellers, readers, and writers? A meta-analytic review
• Evaluation of the Incredible Years Teacher Classroom Management Program in a Regular Norwegian School Setting
• A meta-analysis of teachers’ provision of structure in the classroom and students’ academic competence beliefs, engagement, and achievement
• Fostering elementary school children's public speaking skills: A randomized controlled trial
• Evaluation of the Olweus Bullying Prevention Program: A large scale study of U.S. students in grades 3–11
• A Bayesian Approach to Estimating Effect Sizes in Educational Research
• Dual mechanisms of contextual cueing in immersive visual search
• (When) do teacher professional development interventions improve student Achievement? A meta-analysis of 128 high-quality studies
• A Knowledge Mobilization Framework: Toward Evidence-Based Statistical Communication Practices in Education Research

Discussions

• What about Hedges (2007, doi.org/10.3102/1076...)? He describes several different ways of defining SMDs for cluster-randomized experiments, though in practice I've only ever standardized by total vari [bsky, 1 points, 0 comments]

Related

• Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs
• Examining the Rule of Thumb of Not Using Multilevel Modeling: The “Design Effect Smaller Than Two” Rule
• Evaluating Informative Cluster Size in Cluster Randomized Trials
• A tutorial on conducting sample size and power calculations for detecting treatment effect heterogeneity in cluster randomized trials with linear mixed models
• Correlational effect size benchmarks.