2025/05/06 by Justin Ho, Ho, Justin, Jing Min +1
Decision Sciences · Mathematics · Psychology · #Advanced Causal Inference Techniques #Behavioral and Psychological Studies #Econometrics (econ.EM) #FOS: Computer and information sciences #FOS: Economics and business #Methodology (stat.ME) #Optimal Experimental Design Methods
paper · pdf · doi:10.48550/arxiv.2505.03937
openalex publication_date 2025/05/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Experimental designs are fundamental for estimating causal effects. In some fields, within-subjects designs, which expose participants to both control and treatment at different time periods, are used to address practical and logistical concerns. Counterbalancing, a common technique in within-subjects designs, aims to remove carryover effects by randomizing treatment sequences. Despite its appeal, counterbalancing relies on the assumption that carryover effects are symmetric and cancel out, which is often unverifiable a priori. In this paper, we formalize the challenges of counterbalanced within-subjects designs using the potential outcomes framework. We introduce sequential exchangeability as an additional identification assumption necessary for valid causal inference in these designs. To address identification concerns, we propose diagnostic checks, the use of washout periods, and covariate adjustments, and alternative experimental designs to counterbalanced within-subjects design. Our findings demonstrate the limitations of counterbalancing and provide guidance on when and how within-subjects designs can be appropriately used for causal inference.