Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments

2016/12/16 by Jingwei Zhang, Zhang, Jingwei, Jost Tobias Springenberg +5 · 9 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Optimization and Search Problems #Reinforcement Learning in Robotics #Robotic Path Planning Algorithms #Robotics (cs.RO) #cs.AI #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.1612.05533

Camera ready version for IROS 2017

openalex publication_date 2016/12/16 · openalex created_date 2017/01/06 · arxiv created 2017/07/23 · arxiv updated 2017/07/25 · openalex updated_date 2026/07/28

Abstract

In this paper we consider the problem of robot navigation in simple maze-like environments where the robot has to rely on its onboard sensors to perform the navigation task. In particular, we are interested in solutions to this problem that do not require localization, mapping or planning. Additionally, we require that our solution can quickly adapt to new situations (e.g., changing navigation goals and environments). To meet these criteria we frame this problem as a sequence of related reinforcement learning tasks. We propose a successor feature based deep reinforcement learning algorithm that can learn to transfer knowledge from previously mastered navigation tasks to new problem instances. Our algorithm substantially decreases the required learning time after the first task instance has been solved, which makes it easily adaptable to changing environments. We validate our method in both simulated and real robot experiments with a Robotino and compare it to a set of baseline methods including classical planning-based navigation.

Citations

• Dueling Network Architectures for Deep Reinforcement Learning
• Deep Successor Reinforcement Learning
• Successor Features for Transfer in Reinforcement Learning
• Policy Distillation

Cited by

• Don't Just Search, Understand: Semantic Path Planning Agent for Spherical Tensegrity Robots in Unknown Environments
• Neural SLAM: Learning to Explore with External Memory
• Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
• Universal Successor Features Approximators
• CogniPlan: Uncertainty-Guided Path Planning with Conditional Generative Layout Prediction
• Transfer with Model Features in Reinforcement Learning
• The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation
• MAGIK: Mapping to Analogous Goals via Imagination-enabled Knowledge Transfer
• ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning

Related

• Arena-Rosnav: Towards Deployment of Deep-Reinforcement-Learning-Based Obstacle Avoidance into Conventional Autonomous Navigation Systems
• Robot Navigation with Map-Based Deep Reinforcement Learning
• Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning
• End-to-End Navigation in Unknown Environments using Neural Networks
• NavRep: Unsupervised Representations for Reinforcement Learning of Robot Navigation in Dynamic Human Environments