2020/08/19 by Yifan Zhou, Zhou, Yifan, Yang Liu +5 · 3 citations
Mathematics · #Advanced Causal Inference Techniques #Artificial intelligence #Cluster (spacecraft) #Computer science #Computerized adaptive testing #Estimator #FOS: Computer and information sciences #Mahalanobis distance #Mathematics #Methodology (stat.ME) #Orthogonal array testing #Product (mathematics) #Statistical Methods and Inference #Statistical Methods in Clinical Trials #Statistics #Variance (accounting) #stat.ME
paper · pdf · doi:10.48550/arxiv.2008.08648
published in arXiv (Cornell University) (Cornell University) · 25 pages
arxiv created 2020/08/19 · openalex publication_date 2020/08/19 · arxiv updated 2020/08/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
A/B testing is an important decision-making tool in product development for evaluating user engagement or satisfaction from a new service, feature or product. The goal of A/B testing is to estimate the average treatment effects (ATE) of a new change, which becomes complicated when users are interacting. When the important assumption of A/B testing, the Stable Unit Treatment Value Assumption (SUTVA), which states that each individual's response is affected by their own treatment only, is not valid, the classical estimate of the ATE usually leads to a wrong conclusion. In this paper, we propose a cluster-adaptive network A/B testing procedure, which involves a sequential cluster-adaptive randomization and a cluster-adjusted estimator. The cluster-adaptive randomization is employed to minimize the cluster-level Mahalanobis distance within the two treatment groups, so that the variance of the estimate of the ATE can be reduced. In addition, the cluster-adjusted estimator is used to eliminate the bias caused by network interference, resulting in a consistent estimation for the ATE. Numerical studies suggest our cluster-adaptive network A/B testing achieves consistent estimation with higher efficiency. An empirical study is conducted based on a real world network to illustrate how our method can benefit decision-making in application.