2017/10/10 by James D. Wilson, Wilson, James D., David Uminsky +1
Mathematics · Physics and Astronomy · #Advanced Causal Inference Techniques #Complex Network Analysis Techniques #FOS: Computer and information sciences #Opinion Dynamics and Social Influence #Social and Information Networks (cs.SI)
paper · pdf · doi:10.48550/arxiv.1710.03855
openalex publication_date 2017/10/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we address the fundamental statistical question: how can you assess the power of an A/B test when the units in the study are exposed to interference? This question is germane to many scientific and industrial practitioners that rely on A/B testing in environments where control over interference is limited. We begin by proving that interference has a measurable effect on its sensitivity, or power. We quantify the power of an A/B test of equality of means as a function of the number of exposed individuals under any interference mechanism. We further derive a central limit theorem for the number of exposed individuals under a simple Bernoulli switching interference mechanism. Based on these results, we develop a strategy to estimate the power of an A/B test when actors experience interference according to an observed network model. We demonstrate how to leverage this theory to estimate the power of an A/B test on units sharing any network relationship, and highlight the utility of our method on two applications - a Facebook friendship network as well as a large Twitter follower network. These results yield, for the first time, the capacity to understand how to design an A/B test to detect, with a specified confidence, a fixed measurable treatment effect when the A/B test is conducted under interference driven by networks.