2011/03/16 by Yaming Yu, Yu, Yaming
Business, Management and Accounting · Decision Sciences · Mathematics · #60E15 #62C10 #62L05 #62L15 #Advanced Bandit Algorithms Research #Auction Theory and Applications #Consumer Market Behavior and Pricing #FOS: Mathematics #Probability (math.PR) #Statistics Theory (math.ST) #math.PR #math.ST #msc:60E15 #msc:62C10 #msc:62L05 #msc:62L15 #stat.TH
paper · pdf · doi:10.48550/arxiv.1103.3089
arxiv created 2011/03/16 · openalex publication_date 2011/03/16 · arxiv updated 2011/03/29 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
We study a bandit problem where observations from each arm have an exponential family distribution and different arms are assigned independent conjugate priors. At each of n stages, one arm is to be selected based on past observations. The goal is to find a strategy that maximizes the expected discounted sum of the n observations. Two structural results hold in broad generality: (i) for a fixed prior weight, an arm becomes more desirable as its prior mean increases; (ii) for a fixed prior mean, an arm becomes more desirable as its prior weight decreases. These generalize and unify several results in the literature concerning specific problems including Bernoulli and normal bandits. The second result captures an aspect of the exploration-exploitation dilemma in precise terms: given the same immediate payoff, the less one knows about an arm, the more desirable it becomes because there remains more information to be gained when selecting that arm. For Bernoulli and normal bandits we also obtain extensions to nonconjugate priors.