Meta-learning with Stochastic Linear Bandits

2020/05/18 by Leonardo Cella, Cella, Leonardo, Alessandro Lazaric +3 · 3 citations
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Data Stream Mining Techniques #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2005.08531

Abstract

We investigate meta-learning procedures in the setting of stochastic linear bandits tasks. The goal is to select a learning algorithm which works well on average over a class of bandits tasks, that are sampled from a task-distribution. Inspired by recent work on learning-to-learn linear regression, we consider a class of bandit algorithms that implement a regularized version of the well-known OFUL algorithm, where the regularization is a square euclidean distance to a bias vector. We first study the benefit of the biased OFUL algorithm in terms of regret minimization. We then propose two strategies to estimate the bias within the learning-to-learn setting. We show both theoretically and experimentally, that when the number of tasks grows and the variance of the task-distribution is small, our strategies have a significant advantage over learning the tasks in isolation.

Citations

• Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges.

Cited by

• Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Bandits
• Empirical Bayesian Multi-Bandit Learning
• Secure Best Arm Identification in the Presence of a Copycat

Related

• Meta Representation Learning with Contextual Linear Bandits
• Meta-Learning for Simple Regret Minimization
• Learning-to-Learn Stochastic Gradient Descent with Biased Regularization
• Meta-Learning Adversarial Bandit Algorithms
• Meta-Thompson Sampling