Selecting a Within- or Between-Subject Design for Mediation: Validity, Causality, and Statistical Power

2022/06/09 by Amanda Kay Montoya, Amanda K. Montoya · 68 citations
Computer Science · Mathematics · Psychology · #Advanced Causal Inference Techniques #Advanced Statistical Modeling Techniques #Artificial intelligence #Behavioral and Psychological Studies #Causal inference #Causality (physics) #Computer science #Econometrics #External validity #Inference #Mathematics #Mediation #Psychology #Sample size determination #Statistical inference #Statistical power #Statistics #Subject (documents)

paper · open access · doi:10.1080/00273171.2022.2077287

published in Multivariate Behavioral Research 58(3), 616-636 (Taylor & Francis)

openalex publication_date 2022/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Researchers with mediation hypotheses must consider which design to use: within-subject or between-subject? In this paper, I argue that three factors should influence design choice: validity, causality, and statistical power. Threats to validity include carry-over effects, participant awareness, measurement, and more. Causality is a core element of mediation, and the assumptions required for causal inference differ between the two designs. Between-subject designs require more restrictive no-confounder assumptions, but within-subject designs require the assumption of no carry-over effects. Statistical power should be higher in within-subject designs, but the degree and conditions of this advantage are unknown for mediation analysis. A Monte Carlo simulation compares designs under a broad range of sample sizes, effect sizes, and correlations among repeated measurements. The results show within-subject designs require about half the sample size of between-subject designs to detect indirect effects of the same size, but this difference can vary with population parameters. I provide an empirical example and R script for conducting power analysis for within-subject mediation analysis. Researchers interested in conducting mediation analysis should not select within-subject designs merely because of higher power, but they should also consider validity and causality in their decision, both of which can favor between-subject designs.

Citations

• The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.

Cited by

• Effects of oxytocin administration on non-social executive functions in humans: a preregistered systematic review and meta-analysis
• Message source effects on rejection and costly punishment of criticism across cultures
• Ties of Trust: a bowtie model to uncover trustor-trustee relationships in LLMs
• Empowering leadership during the COVID-19 outbreak: Implications for work satisfaction and effectiveness in organizational teams. [europepmc]
• Adaptive Intervention for School-Age, Minimally Verbal Children With Autism Spectrum Disorder in the Community: Primary Aim Results. [europepmc]
• Message source effects on rejection and costly punishment of criticism across cultures. [europepmc]
• Aligning early childhood science teaching beliefs, practices, and children's learning outcomes: the impact of a professional development program. [europepmc]
• Intermediate endophenotypes and epigenetic mechanisms of psychiatric disorders. [europepmc]
• ML-based validation of experimental randomization in learning games. [europepmc]
• Biomechanical mechanisms underlying the effect of minimalist footwear on walking stability in persons with a history of falls. [europepmc]
• Reflexive priority of biological motion in attentional orienting: evidence from conflict resistance in complex visual settings [europepmc]

Related

• Required Sample Size to Detect the Mediated Effect
• Survey Design, Order Effects, and Causal Mediation Analysis
• Clarifying causal mediation analysis for the applied researcher: Defining effects based on what we want to learn
• Time and Other Considerations in Mediation Design
• Power of Mediation Effects Using Bootstrap Resampling