Eligibility Propagation to Speed up Time Hopping for Reinforcement Learning

2009/04/03 by Petar Kormushev, Kormushev, Petar, Kohei Nomoto +5
Computer Science · #Adaptive Dynamic Programming Control #Artificial Intelligence (cs.AI) #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robotics (cs.RO) #cs.AI #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.0904.0546

7 pages

arxiv created 2009/04/03 · openalex publication_date 2009/04/03 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

A mechanism called Eligibility Propagation is proposed to speed up the Time Hopping technique used for faster Reinforcement Learning in simulations. Eligibility Propagation provides for Time Hopping similar abilities to what eligibility traces provide for conventional Reinforcement Learning. It propagates values from one state to all of its temporal predecessors using a state transitions graph. Experiments on a simulated biped crawling robot confirm that Eligibility Propagation accelerates the learning process more than 3 times.

Related

• Time Hopping technique for faster reinforcement learning in simulations
• A Unified Approach for Multi-step Temporal-Difference Learning with Eligibility Traces in Reinforcement Learning
• A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning
• Learning Transition Models with Time-delayed Causal Relations
• Temporal Abstraction in Reinforcement Learning with Offline Data