Nonparametric identification is not enough, but randomized controlled trials are

2021/08/25 by P. M. Aronow, Peter M. Aronow, James M. Robins +9 · 5 voices · 9 citations
Mathematics · Medicine · #Advanced Causal Inference Techniques #Artificial intelligence #Average treatment effect #Causal inference #Computer science #Confidence interval #Confounding #Covariate #Econometrics #Estimator #Inference #Mathematics #Medicine #Nonparametric statistics #Observational study #Propensity score matching #Randomized controlled trial #Randomized experiment #Sample size determination #Statistical Methods and Inference #Statistical Methods in Clinical Trials #Statistical inference #Statistics #stat.ME

paper · pdf · doi:10.48550/arxiv.2108.11342

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2021/08/25 · arxiv created 2021/09/27 · arxiv updated 2021/09/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We argue that randomized controlled trials (RCTs) are special even among settings where average treatment effects are identified by a nonparametric unconfoundedness assumption. This claim follows from two results of Robins and Ritov (1997): (1) with at least one continuous covariate control, no estimator of the average treatment effect exists which is uniformly consistent without further assumptions, (2) knowledge of the propensity score yields a uniformly consistent estimator and honest confidence intervals that shrink at parametric rates with increasing sample size, regardless of how complicated the propensity score function is. We emphasize the latter point, and note that successfully-conducted RCTs provide knowledge of the propensity score to the researcher. We discuss modern developments in covariate adjustment for RCTs, noting that statistical models and machine learning methods can be used to improve efficiency while preserving finite sample unbiasedness. We conclude that statistical inference has the potential to be fundamentally more difficult in observational settings than it is in RCTs, even when all confounders are measured.

Citations

• What can be estimated? Identifiability, estimability, causal inference and ill-posed inverse problems
• Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score

Cited by

• Unifying Statistical and Mathematical Modeling Through a Causal Inference Lens
• Nonparametric regression for cost-effectiveness analyses with observational data -- a tutorial
• Constructing g-computation estimators: two case studies in selection bias
• PrivATE: Differentially Private Confidence Intervals for Average Treatment Effects

Discussions

• Yes! And then they wrote this to show why Pearl was also technically wrong. arxiv.org/abs/2108.11342 [bsky, 17 points, 1 comments]
• Here's a cool back and forth. Original paper from Aronow et al.: arxiv.org/pdf/2108.11342 Response from Dimmery and Munger: arxiv.org/pdf/2501.12161 [bsky, 12 points, 2 comments]
• Aronow et al 2022 is a clear statement arxiv.org/abs/2108.11342 for the superiority claim and the argument for it does what you're looking for, I think. [bsky, 9 points, 1 comments]
• If you haven't read this one, it's a similar criticism but a bit shorter and different approach arxiv.org/abs/2108.11342 [bsky, 5 points, 1 comments]
• arxiv.org/abs/2108.11342 [bsky, 0 points, 0 comments]

Related

• Understanding and misunderstanding randomized controlled trials
• Minimax Rates and Adaptivity in Combining Experimental and Observational Data
• What Makes Treatment Effects Identifiable? Characterizations and Estimators Beyond Unconfoundedness
• Variance Reduction for Causal Inference
• Mostly Harmless Simulations? Using Monte Carlo Studies for Estimator Selection