A Two-armed Bandit Framework for A/B Testing

2025/07/24 by Jinjuan Wang, Wang, Jinjuan, Yu Zhang +6
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Adversarial Robustness in Machine Learning #Applications (stat.AP) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2507.18118

openalex publication_date 2025/07/24 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

A/B testing is widely used in modern technology companies for policy evaluation and product deployment, with the goal of comparing the outcomes under a newly-developed policy against a standard control. Various causal inference and reinforcement learning methods developed in the literature are applicable to A/B testing. This paper introduces a two-armed bandit framework designed to improve the power of existing approaches. The proposed procedure consists of three main steps: (i) employing doubly robust estimation to generate pseudo-outcomes, (ii) utilizing a two-armed bandit framework to construct the test statistic, and (iii) applying a permutation-based method to compute the p-value. We demonstrate the efficacy of the proposed method through asymptotic theories, numerical experiments and real-world data from a ridesharing company, showing its superior performance in comparison to existing methods.

Citations

• Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach
• Statistical Inference in Reinforcement Learning: A Selective Survey
• ARMA-Design: Optimal Treatment Allocation Strategies for A/B Testing in Partially Observable Time Series Experiments
• Improving the Estimation of Lifetime Effects in A/B Testing via Treatment Locality
• Data-Driven Switchback Experiments: Theoretical Tradeoffs and Empirical Bayes Designs
• Combining Experimental and Historical Data for Policy Evaluation
• Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments
• Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Making
• Causal clustering: design of cluster experiments under network interference
• A/B Testing: A Systematic Literature Review
• Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharing
• An Efficient Doubly-Robust Test for the Kernel Treatment Effect
• Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology
• A Review of Off-Policy Evaluation in Reinforcement Learning
• Anytime-valid off-policy inference for contextual bandits
• Markovian Interference in Experiments
• Strategy-Driven Limit Theorems Associated Bandit Problems
• Policy Evaluation for Temporal and/or Spatial Dependent Experiments
• Multiple Randomization Designs: Estimation and Inference with Interference
• Rate-Optimal Cluster-Randomized Designs for Spatial Interference
• Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning
• Average Direct and Indirect Causal Effects under Interference
• Graph-Based Equilibrium Metrics for Dynamic Supply-Demand Systems with Applications to Ride-sourcing Platforms
• Graph-Based Equilibrium Metrics for Dynamic Supply–Demand Systems With Applications to Ride-sourcing Platforms
• Trustworthy Online Marketplace Experimentation with Budget-split Design
• Debiased Inverse Propensity Score Weighting for Estimation of Average Treatment Effects with High-Dimensional Confounders
• Toward Better Practice of Covariate Adjustment in Analyzing Randomized Clinical Trials
• Design and Analysis of Switchback Experiments
• Batch Policy Learning in Average Reward Markov Decision Processes
• Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score
• Adaptive Experimental Design with Temporal Interference: A Maximum\n Likelihood Approach
• Testing Mediation Effects Using Logic of Boolean Matrices
• Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework
• Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings
• Ridesourcing systems: A framework and review
• Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning
• Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching
• Batch Policy Learning under Constraints
• Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
• Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures
• Counterfactual Mean Embeddings
• Policy Evaluation and Optimization with Continuous Treatments
• Double/Debiased/Neyman Machine Learning of Treatment Effects
• Double/Debiased/Neyman Machine Learning of Treatment Effects
• Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning
• Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection
• Randomization and The Pernicious Effects of Limited Budgets on Auction Experiments
• Approximate Residual Balancing: De-Biased Inference of Average Treatment\n Effects in High Dimensions
• Always Valid Inference: Bringing Sequential Analysis to A/B Testing
• Doubly Robust Off-policy Value Evaluation for Reinforcement Learning
• Doubly Robust Policy Evaluation and Optimization
• Balancing Covariates via Propensity Score Weighting
• Graph cluster randomization: network exposure to multiple universes
• Bias-Corrected Matching Estimators for Average Treatment Effects
• On the Validity of Covariate Adjustment for Estimating Causal Effects
• P-values for high-dimensional regression
• Doubly Robust Estimation in Missing Data and Causal Inference Models
• The central role of the propensity score in observational studies for causal effects
• Smoothing noisy data with spline functions
• Reinforcement Learning: An Introduction

Related

• Strategic A/B testing via Maximum Probability-driven Two-armed Bandit
• Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework
• ARMA-Design: Optimal Treatment Allocation Strategies for A/B Testing in Partially Observable Time Series Experiments
• Efficient Counterfactual Learning from Bandit Feedback
• A Framework for Fair Evaluation of Variance-Aware Bandit Algorithms