Reward-Biased Maximum Likelihood Estimation for Linear Stochastic Bandits

2020/10/08 by Yu-Heng Hung, Hung, Yu-Heng, Ping-Chun Hsieh +5
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Smart Grid Energy Management

paper · pdf · doi:10.48550/arxiv.2010.04091

openalex publication_date 2020/10/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Modifying the reward-biased maximum likelihood method originally proposed in the adaptive control literature, we propose novel learning algorithms to handle the explore-exploit trade-off in linear bandits problems as well as generalized linear bandits problems. We develop novel index policies that we prove achieve order-optimality, and show that they achieve empirical performance competitive with the state-of-the-art benchmark methods in extensive experiments. The new policies achieve this with low computation time per pull for linear bandits, and thereby resulting in both favorable regret as well as computational efficiency.

Citations

• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Online Stochastic Linear Optimization under One-bit Feedback
• Bayesian bandits: balancing the exploration-exploitation tradeoff via double sampling

Related

• Exploration Through Reward Biasing: Reward-Biased Maximum Likelihood Estimation for Stochastic Multi-Armed Bandits
• Reward Biased Maximum Likelihood Estimation for Reinforcement Learning
• Reward-Biased Maximum Likelihood Estimation for Neural Contextual Bandits
• Asymptotically Optimal Bandits under Weighted Information
• Exploration via linearly perturbed loss minimisation