Bayesian Counterfactual Risk Minimization

2018/06/29 by Ben London, London, Ben, Ted Sandler +1 · 1 citation
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Machine Learning and Algorithms #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1806.11500

Abstract

We present a Bayesian view of counterfactual risk minimization (CRM) for offline learning from logged bandit feedback. Using PAC-Bayesian analysis, we derive a new generalization bound for the truncated inverse propensity score estimator. We apply the bound to a class of Bayesian policies, which motivates a novel, potentially data-dependent, regularization technique for CRM. Experimental results indicate that this technique outperforms standard L₂ regularization, and that it is competitive with variance regularization while being both simpler to implement and more computationally efficient.

Citations

• Logistic-normal distributions:Some properties and uses
• The central role of the propensity score in observational studies for causal effects

Cited by

• Offline Contextual Bandit with Counterfactual Sample Identification

Related

• Sequential Counterfactual Risk Minimization
• Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
• Efficient Counterfactual Learning from Bandit Feedback
• PAC-Bayesian Offline Contextual Bandits With Guarantees
• Exponential Smoothing for Off-Policy Learning