Risk-averse Contextual Multi-armed Bandit Problem with Linear Payoffs

2022/06/24 by Yi‐Fan Lin, Yuhao Wang, Lin, Yifan +3 · 1 citation
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Search Problems #Risk and Portfolio Optimization

paper · pdf · doi:10.48550/arxiv.2206.12463

openalex publication_date 2022/06/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we consider the contextual multi-armed bandit problem for linear payoffs under a risk-averse criterion. At each round, contexts are revealed for each arm, and the decision maker chooses one arm to pull and receives the corresponding reward. In particular, we consider mean-variance as the risk criterion, and the best arm is the one with the largest mean-variance reward. We apply the Thompson Sampling algorithm for the disjoint model, and provide a comprehensive regret analysis for a variant of the proposed algorithm. For T rounds, K actions, and d-dimensional feature vectors, we prove a regret bound of O((1+ρ+\frac1ρ) dln T ln \fracKδ√d K T^1+2ε ln \fracKδ \frac1ε) that holds with probability 1-δ under the mean-variance criterion with risk tolerance ρ, for any 0<ε<(1)/(2), 0<δ<1. The empirical performance of our proposed algorithms is demonstrated via a portfolio selection problem.

Cited by

• Online Bayesian Risk-Averse Reinforcement Learning

Related

• Generalized Risk-Aversion in Stochastic Multi-Armed Bandits
• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Thompson Sampling Algorithms for Mean-Variance Bandits
• Doubly robust Thompson sampling for linear payoffs
• Analysis of Thompson Sampling for the multi-armed bandit problem