2018/03/23 by Craig Sherstan, Marlos C. Machado, Sherstan, Craig +3
Computer Science · #Reinforcement Learning in Robotics #Machine Learning and Data Classification #Machine Learning and Algorithms
paper · pdf · doi:10.48550/arxiv.1803.09001
Here we propose using the successor representation (SR) to accelerate\nlearning in a constructive knowledge system based on general value functions\n(GVFs). In real-world settings like robotics for unstructured and dynamic\nenvironments, it is infeasible to model all meaningful aspects of a system and\nits environment by hand due to both complexity and size. Instead, robots must\nbe capable of learning and adapting to changes in their environment and task,\nincrementally constructing models from their own experience. GVFs, taken from\nthe field of reinforcement learning (RL), are a way of modeling the world as\npredictive questions. One approach to such models proposes a massive network of\ninterconnected and interdependent GVFs, which are incrementally added over\ntime. It is reasonable to expect that new, incrementally added predictions can\nbe learned more swiftly if the learning process leverages knowledge gained from\npast experience. The SR provides such a means of separating the dynamics of the\nworld from the prediction targets and thus capturing regularities that can be\nreused across multiple GVFs. As a primary contribution of this work, we show\nthat using SR-based predictions can improve sample efficiency and learning\nspeed in a continual learning setting where new predictions are incrementally\nadded and learned over time. We analyze our approach in a grid-world and then\ndemonstrate its potential on data from a physical robot arm.\n