CURLing the Dream: Contrastive Representations for World Modeling in Reinforcement Learning

2024/08/11 by Victor Augusto Kich, Kich, Victor Augusto, Jair Augusto Bottega +9
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2408.05781

openalex publication_date 2024/08/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we present Curled-Dreamer, a novel reinforcement learning algorithm that integrates contrastive learning into the DreamerV3 framework to enhance performance in visual reinforcement learning tasks. By incorporating the contrastive loss from the CURL algorithm and a reconstruction loss from autoencoder, Curled-Dreamer achieves significant improvements in various DeepMind Control Suite tasks. Our extensive experiments demonstrate that Curled-Dreamer consistently outperforms state-of-the-art algorithms, achieving higher mean and median scores across a diverse set of tasks. The results indicate that the proposed approach not only accelerates learning but also enhances the robustness of the learned policies. This work highlights the potential of combining different learning paradigms to achieve superior performance in reinforcement learning applications.

Related

• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations
• Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction
• Dream to Control: Learning Behaviors by Latent Imagination
• DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction