Representation-Driven Reinforcement Learning

2023/05/31 by Nabati, Ofir, Tennenholtz, Guy, Mannor, Shie · 2 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2305.19922

Abstract

We present a representation-driven framework for reinforcement learning. By representing policies as estimates of their expected values, we leverage techniques from contextual bandits to guide exploration and exploitation. Particularly, embedding a policy network into a linear feature space allows us to reframe the exploration-exploitation problem as a representation-exploitation problem, where good policy representations enable optimal exploration. We demonstrate the effectiveness of this framework through its application to evolutionary and policy gradient-based approaches, leading to significantly improved performance compared to traditional methods. Our framework provides a new perspective on reinforcement learning, highlighting the importance of policy representation in determining optimal exploration-exploitation strategies.

Cited by

• Reinforcement Learning with Discrete Diffusion Policies for Combinatorial Action Spaces
• Spectral Bellman Method: Unifying Representation and Exploration in RL

Related

• Learning Actionable Representations with Goal-Conditioned Policies
• Explicit Explore-Exploit Algorithms in Continuous State Spaces
• Reward-Conditioned Policies
• Discovering Reinforcement Learning Algorithms
• Learning Action Representations for Reinforcement Learning