2018/04/12 by Dimitri P. Bertsekas, Bertsekas, Dimitri P. · 3 citations
Computer Science · #49 #90 #93 #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.1804.04577
openalex publication_date 2018/04/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper we discuss policy iteration methods for approximate solution of\na finite-state discounted Markov decision problem, with a focus on\nfeature-based aggregation methods and their connection with deep reinforcement\nlearning schemes. We introduce features of the states of the original problem,\nand we formulate a smaller "aggregate" Markov decision problem, whose states\nrelate to the features. We discuss properties and possible implementations of\nthis type of aggregation, including a new approach to approximate policy\niteration. In this approach the policy improvement operation combines\nfeature-based aggregation with feature construction using deep neural networks\nor other calculations. We argue that the cost function of a policy may be\napproximated much more accurately by the nonlinear function of the features\nprovided by aggregation, than by the linear function of the features provided\nby neural network-based reinforcement learning, thereby potentially leading to\nmore effective policy improvement.\n