Risk-Aversion in Multi-armed Bandits

2013/01/09 by Amir Sani, Sani, Amir, Alessandro Lazaric +3 · 10 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Auction Theory and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1301.1936

openalex publication_date 2013/01/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Stochastic multi-armed bandits solve the Exploration-Exploitation dilemma and ultimately maximize the expected reward. Nonetheless, in many practical problems, maximizing the expected reward is not the most desirable objective. In this paper, we introduce a novel setting based on the principle of risk-aversion where the objective is to compete against the arm with the best risk-return trade-off. This setting proves to be intrinsically more difficult than the standard multi-arm bandit setting due in part to an exploration risk which introduces a regret associated to the variability of an algorithm. Using variance as a measure of risk, we introduce two new algorithms, investigate their theoretical guarantees, and report preliminary empirical results.

Cited by

• Optimal Best-Arm Identification Methods for Tail-Risk Measures
• Adaptive Sampling for Stochastic Risk-Averse Learning
• Risk-Averse Action Selection Using Extreme Value Theory Estimates of the\n CVaR
• Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions
• Risk averse non-stationary multi-armed bandits
• Distribution oblivious, risk-aware algorithms for multi-armed bandits with unbounded rewards
• Index-Based Policy for Risk-Averse Multi-Armed Bandit
• A Reliability-aware Multi-armed Bandit Approach to Learn and Select Users in Demand Response
• Risk-Constrained Thompson Sampling for CVaR Bandits
• Task Recommendation in Crowdsourcing Based on Learning Preferences and Reliabilities

Related

• Optimal Exploration-Exploitation in a Multi-Armed-Bandit Problem with\n Non-stationary Rewards
• Index-Based Policy for Risk-Averse Multi-Armed Bandit
• Generalized Risk-Aversion in Stochastic Multi-Armed Bandits
• Multiarmed Bandits Problem Under the Mean-Variance Setting
• Risk-sensitive Bandits: Arm Mixture Optimality and Regret-efficient Algorithms