On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning

2019/08/22 by Tao Li, Quanyan Zhu, Li, Tao +1 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Gene Regulatory Network Analysis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1908.08578

openalex publication_date 2019/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions.

Cited by

• Stackelberg Coupling of Online Representation Learning and Reinforcement Learning
• Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigation

Related

• Adaptive Online Value Function Approximation with Wavelets
• Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimension
• Adaptive Bases for Reinforcement Learning
• Adaptive Resolving Methods for Reinforcement Learning with Function Approximations
• Unsupervised Basis Function Adaptation for Reinforcement Learning