Joint AP Probing and Scheduling: A Contextual Bandit Approach

2021/08/06 by Tianyi Xu, Xu, Tianyi, Ding Zhang +5 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing #Networking and Internet Architecture (cs.NI) #Optimization and Search Problems

paper · pdf · doi:10.48550/arxiv.2108.03297

openalex publication_date 2021/08/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider a set of APs with unknown data rates that cooperatively serve a mobile client. The data rate of each link is i.i.d. sampled from a distribution that is unknown a priori. In contrast to traditional link scheduling problems under uncertainty, we assume that in each time step, the device can probe a subset of links before deciding which one to use. We model this problem as a contextual bandit problem with probing (CBwP) and present an efficient algorithm. We further establish the regret of our algorithm for links with Bernoulli data rates. Our CBwP model is a novel extension of the classic contextual bandit model and can potentially be applied to a large class of sequential decision-making problems that involve joint probing and play under uncertainty.

Cited by

• Online Learning with Probing for Sequential User-Centric Selection
• Fair Algorithms with Probing for Multi-Agent Multi-Armed Bandits

Related

• Contextual Blocking Bandits
• Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks
• Stochastic Contextual Bandits with Known Reward Functions
• Regret of Age-of-Information Bandits
• AoI-based Scheduling of Correlated Sources for Timely Inference