Two-Armed Restless Bandits with Imperfect Information: Stochastic\n Control and Indexability

2015/06/24 by Roland G. Fryer, Philipp Harms, Fryer, Roland +1 · 1 citation
Decision Sciences · Economics, Econometrics and Finance · Engineering · #60G40 (Secondary) #93E11 #93E20 (Primary) #Advanced Bandit Algorithms Research #Auction Theory and Applications #FOS: Mathematics #Financial Markets and Investment Strategies #Optimization and Control (math.OC) #Smart Grid Energy Management

paper · pdf · doi:10.48550/arxiv.1506.07291

openalex publication_date 2015/06/24 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28

Abstract

We present a two-armed bandit model of decision making under uncertainty\nwhere the expected return to investing in the "risky arm" increases when\nchoosing that arm and decreases when choosing the "safe" arm. These dynamics\nare natural in applications such as human capital development, job search, and\noccupational choice. Using new insights from stochastic control, along with a\nmonotonicity condition on the payoff dynamics, we show that optimal strategies\nin our model are stopping rules that can be characterized by an index which\nformally coincides with Gittins' index. Our result implies the indexability of\na new class of restless bandit models.\n

Citations

• A two-armed bandit theory of market pricing
• The Nonstochastic Multiarmed Bandit Problem

Cited by

• Threshold-indexability of restless bandits with real interval state spaces: a performance-metric verification framework and long-run average analysis

Related

• Indexability and Rollout Policy for Multi-State Partially Observable Restless Bandits
• Gittins' theorem under uncertainty
• Stochastic Multi-armed Bandits with Non-stationary Rewards Generated by a Linear Dynamical System
• On the Whittle Index for Restless Multi-armed Hidden Markov Bandits
• Restless Bandits with Constrained Arms: Applications in Social and Information Networks