2022/12/15 by Dimitri P. Bertsekas, Bertsekas, Dimitri
Decision Sciences · Economics, Econometrics and Finance · Engineering · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #Economic theories and models #FOS: Computer and information sciences #FOS: Electrical engineering #Systems and Control (eess.SY) #Water resources management and optimization #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2212.07998
openalex publication_date 2022/12/15 · openalex created_date 2023/01/04 · openalex updated_date 2026/07/28
We provide a unifying approximate dynamic programming framework that applies to a broad variety of problems involving sequential estimation. We consider first the construction of surrogate cost functions for the purposes of optimization, and we focus on the special case of Bayesian optimization, using the rollout algorithm and some of its variations. We then discuss the more general case of sequential estimation of a random vector using optimal measurement selection, and its application to problems of stochastic and adaptive control. We distinguish between adaptive control of deterministic and stochastic systems: the former are better suited for the use of rollout, while the latter are well suited for the use of rollout with certainty equivalence approximations. As an example of the deterministic case, we discuss sequential decoding problems, and a rollout algorithm for the approximate solution of the Wordle and Mastermind puzzles, recently developed in the paper [BBB22].