Open-Ended Reinforcement Learning with Neural Reward Functions

2022/02/16 by R. R. Meier, Asier Mujika, Meier, Robert +1
Computer Science · #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Reinforcement Learning in Robotics #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2202.08266

openalex publication_date 2022/02/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Inspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more on unsupervised discovery of skills. Most current approaches, like DIAYN or DADS, optimize some form of mutual information objective. We propose a different approach that uses reward functions encoded by neural networks. These are trained iteratively to reward more complex behavior. In high-dimensional robotic environments our approach learns a wide range of interesting skills including front-flips for Half-Cheetah and one-legged running for Humanoid. In the pixel-based Montezuma's Revenge environment our method also works with minimal changes and it learns complex skills that involve interacting with items and visiting diverse locations. The implementation of our approach can be found in this link: https://github.com/amujika/Open-Ended-Reinforcement-Learning-with-Neural-Reward-Functions.

Related

• Functions that Emerge through End-to-End Reinforcement Learning - The Direction for Artificial General Intelligence -
• Learning as Reinforcement: Applying Principles of Neuroscience for More General Reinforcement Learning Agents
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• End-to-End Robotic Reinforcement Learning without Reward Engineering
• On-Robot Reinforcement Learning with Goal-Contrastive Rewards