2008/11/05 by Sahand Haji Ali Ahmad, Ahmad, Sahand H. A., Mingyan Liu +7
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Age of Information Optimization #Cognitive Radio Networks and Spectrum Sensing #FOS: Computer and information sciences #Information Theory (cs.IT) #Networking and Internet Architecture (cs.NI)
paper · pdf · doi:10.48550/arxiv.0811.0637
openalex publication_date 2008/11/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider opportunistic communications over multiple channels where the state ("good" or "bad") of each channel evolves as independent and identically distributed Markov processes. A user, with limited sensing and access capability, chooses one channel to sense and subsequently access (based on the sensed channel state) in each time slot. A reward is obtained when the user senses and accesses a "good" channel. The objective is to design the optimal channel selection policy that maximizes the expected reward accrued over time. This problem can be generally cast as a Partially Observable Markov Decision Process (POMDP) or a restless multi-armed bandit process, to which optimal solutions are often intractable. We show in this paper that the myopic policy, with a simple and robust structure, achieves optimality under certain conditions. This result finds applications in opportunistic communications in fading environment, cognitive radio networks for spectrum overlay, and resource-constrained jamming and anti-jamming.