The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy

2020/10/08 by Masahiro Kato, Kato, Masahiro, Shota Yasui +3 · 1 citation
Computer Science · Decision Sciences · Economics, Econometrics and Finance · Mathematics · #Advanced Statistical Process Monitoring #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #cs.LG #econ.EM #stat.ME #stat.ML

paper · pdf · doi:10.48550/arxiv.2010.03792

arxiv created 2021/06/18 · arxiv updated 2021/06/22

Abstract

The doubly robust (DR) estimator, which consists of two nuisance parameters, the conditional mean outcome and the logging policy (the probability of choosing an action), is crucial in causal inference. This paper proposes a DR estimator for dependent samples obtained from adaptive experiments. To obtain an asymptotically normal semiparametric estimator from dependent samples with non-Donsker nuisance estimators, we propose adaptive-fitting as a variant of sample-splitting. We also report an empirical paradox that our proposed DR estimator tends to show better performances compared to other estimators utilizing the true logging policy. While a similar phenomenon is known for estimators with i.i.d. samples, traditional explanations based on asymptotic efficiency cannot elucidate our case with dependent samples. We confirm this hypothesis through simulation studies.

Citations

• Estimating causal effects of treatments in randomized and nonrandomized studies.
• Doubly Robust Estimation in Missing Data and Causal Inference Models
• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Off-Policy Evaluation and Learning for External Validity under a Covariate Shift
• Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score

Cited by

• Off-Policy Evaluation of Bandit Algorithm from Dependent Samples under\n Batch Update Policy

Related

• Adaptive Doubly Robust Estimator from Non-stationary Logging Policy under a Convergence of Average Probability
• Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits
• Efficient Adaptive Experimentation with Noncompliance
• Doubly Robust Inference for Targeted Minimum Loss Based Estimation in Randomized Trials with Missing Outcome Data
• Statistical Inference for Data-adaptive Doubly Robust Estimators with Survival Outcomes