2020/11/09 by Mukul Gagrani, Sagar Sudhakara, Gagrani, Mukul +7 · 1 citation
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Optimization and Search Problems #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.2011.04686
We consider optimal control of an unknown multi-agent linear quadratic (LQ) system where the dynamics and the cost are coupled across the agents through the mean-field (i.e., empirical mean) of the states and controls. Directly using single-agent LQ learning algorithms in such models results in regret which increases polynomially with the number of agents. We propose a new Thompson sampling based learning algorithm which exploits the structure of the system model and show that the expected Bayesian regret of our proposed algorithm for a system with agents of |M| different types at time horizon T is O ( |M|1.5 √(T) ) irrespective of the total number of agents, where the O notation hides logarithmic factors in T. We present detailed numerical experiments to illustrate the salient features of the proposed algorithm.