Rates of Convergence in Certain Native Spaces of Approximations used in Reinforcement Learning

2023/09/14 by Bouland, Ali, Niu, Shengyuan, Paruchuri, Sai Tej +3
#FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2309.07383

Abstract

This paper studies convergence rates for some value function approximations that arise in a collection of reproducing kernel Hilbert spaces (RKHS) H(Ω). By casting an optimal control problem in a specific class of native spaces, strong rates of convergence are derived for the operator equation that enables offline approximations that appear in policy iteration. Explicit upper bounds on error in value function and controller approximations are derived in terms of power function P_H,N for the space of finite dimensional approximants H_N in the native space H(Ω). These bounds are geometric in nature and refine some well-known, now classical results concerning convergence of approximations of value functions.

Related

• Convergence Rates of Online Critic Value Function Approximation in Native Spaces
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
• Reinforcement Learning with Linear Function Approximation and LQ control Converges
• Tailoring Reproducing Kernels for Optimal Control via Policy Iteration
• On the Convergence of Reinforcement Learning in Nonlinear Continuous State Space Problems