Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning

2025/06/11 by Daniel E. Lawson, Adriana Hugessen, Lawson, Daniel +7 · 3 citations
Computer Science · #Reinforcement Learning in Robotics #Domain Adaptation and Few-Shot Learning #Artificial Intelligence in Games

paper · pdf · doi:10.48550/arxiv.2506.10137

Abstract

While goal-conditioned behavior cloning (GCBC) methods can perform well on in-distribution training tasks, they do not necessarily generalize zero-shot to tasks that require conditioning on novel state-goal pairs, i.e. combinatorial generalization. In part, this limitation can be attributed to a lack of temporal consistency in the state representation learned by BC; if temporally correlated states are properly encoded to similar latent representations, then the out-of-distribution gap for novel state-goal pairs would be reduced. We formalize this notion by demonstrating how encouraging long-range temporal consistency via successor representations (SR) can facilitate generalization. We then propose a simple yet effective representation learning objective, BYOL-γ for GCBC, which theoretically approximates the successor representation in the finite MDP case through self-predictive representations, and achieves competitive empirical performance across a suite of challenging tasks requiring combinatorial generalization.

Cited by

• Action-Sufficient Goal Representations
• Dual Goal Representations
• TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning

Related

• When Does Predictive Inverse Dynamics Outperform Behavior Cloning?
• Bridging State and History Representations: Understanding Self-Predictive RL
• Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learning
• Understanding Self-Predictive Learning for Reinforcement Learning
• Self-Supervised Adversarial Imitation Learning