Learning Generative Models with Goal-conditioned Reinforcement Learning

2023/03/26 by Mariana Vargas Vieyra, Vieyra, Mariana Vargas, Pierre Ménard +1
Computer Science · #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2303.14811

openalex publication_date 2023/03/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

We present a novel, alternative framework for learning generative models with goal-conditioned reinforcement learning. We define two agents, a goal conditioned agent (GC-agent) and a supervised agent (S-agent). Given a user-input initial state, the GC-agent learns to reconstruct the training set. In this context, elements in the training set are the goals. During training, the S-agent learns to imitate the GC-agent while remaining agnostic of the goals. At inference we generate new samples with the S-agent. Following a similar route as in variational auto-encoders, we derive an upper bound on the negative log-likelihood that consists of a reconstruction term and a divergence between the GC-agent policy and the (goal-agnostic) S-agent policy. We empirically demonstrate that our method is able to generate diverse and high quality samples in the task of image synthesis.

Related

• Consistent Zero-Shot Imitation with Contrastive Goal Inference
• Reinforcement Learning for Generative AI: A Survey
• Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning
• Visual Reinforcement Learning with Imagined Goals
• Deep Generative Models with Learnable Knowledge Constraints