2020/09/06 by Gourab Ghatak, Ghatak, Gourab · 1 citation
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing #Signal Processing (eess.SP) #Smart Grid Energy Management #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2009.02791
openalex publication_date 2020/09/06 · openalex created_date 2022/07/23 · openalex updated_date 2026/07/28
We consider a non-stationary two-armed bandit framework and propose a\nchange-detection based Thompson sampling (TS) algorithm, named TS with\nchange-detection (TS-CD), to keep track of the dynamic environment. The\nnon-stationarity is modeled using a Poisson arrival process, which changes the\nmean of the rewards on each arrival. The proposed strategy compares the\nempirical mean of the recent rewards of an arm with the estimate of the mean of\nthe rewards from its history. It detects a change when the empirical mean\ndeviates from the mean estimate by a value larger than a threshold. Then, we\ncharacterize the lower bound on the duration of the time-window for which the\nbandit framework must remain stationary for TS-CD to successfully detect a\nchange when it occurs. Consequently, our results highlight an upper bound on\nthe parameter for the Poisson arrival process, for which the TS-CD achieves\nasymptotic regret optimality with high probability. Finally, we validate the\nefficacy of TS-CD by testing it for edge-control of radio access technique\n(RAT)-selection in a wireless network. Our results show that TS-CD not only\noutperforms the classical max-power RAT selection strategy but also other\nactively adaptive and passively adaptive bandit algorithms that are designed\nfor non-stationary environments.\n