Iterative Model-Based Reinforcement Learning Using Simulations in the Differentiable Neural Computer

2019/06/17 by Adeel Mufti, Mufti, Adeel, Svetlin Penkov +3
Computer Science · Engineering · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1906.07248

openalex publication_date 2019/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a lifelong learning architecture, the Neural Computer Agent (NCA), where a Reinforcement Learning agent is paired with a predictive model of the environment learned by a Differentiable Neural Computer (DNC). The agent and DNC model are trained in conjunction iteratively. The agent improves its policy in simulations generated by the DNC model and rolls out the policy to the live environment, collecting experiences in new portions or tasks of the environment for further learning. Experiments in two synthetic environments show that DNC models can continually learn from pixels alone to simulate new tasks as they are encountered by the agent, while the agents can be successfully trained to solve the tasks using Proximal Policy Optimization entirely in simulations.

Citations

• Recurrent World Models Facilitate Policy Evolution

Related

• Environments for Lifelong Reinforcement Learning
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity
• General policy mapping: online continual reinforcement learning inspired on the insect brain
• On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models
• Deep Reinforcement Learning