2023/09/18 by Jacob Carruth, Maximilian F. Eggl, Carruth, Jacob +5
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Optimization and Control (math.OC) #Target Tracking and Data Fusion in Sensor Networks
paper · pdf · doi:10.48550/arxiv.2309.10138
openalex publication_date 2023/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Here and in a follow-on paper, we consider a simple control problem in which the underlying dynamics depend on a parameter a that is unknown and must be learned. In this paper, we assume that a is bounded, i.e., that |a| ≤ aMAX, and we study two variants of the control problem. In the first variant, Bayesian control, we are given a prior probability distribution for a and we seek a strategy that minimizes the expected value of a given cost function. Assuming that we can solve a certain PDE (the Hamilton-Jacobi-Bellman equation), we produce optimal strategies for Bayesian control. In the second variant, agnostic control, we assume nothing about a and we seek a strategy that minimizes a quantity called the regret. We produce a prior probability distribution dPrior(a) supported on a finite subset of [-aMAX,aMAX] so that the agnostic control problem reduces to the Bayesian control problem for the prior dPrior(a).