Transfer Learning by Modeling a Distribution over Policies

2019/06/09 by Disha Shrivastava, Eeshan Gunesh Dhekane, Shrivastava, Disha +3
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Smart Grid Energy Management

paper · pdf · doi:10.48550/arxiv.1906.03574

openalex publication_date 2019/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Exploration and adaptation to new tasks in a transfer learning setup is a central challenge in reinforcement learning. In this work, we build on the idea of modeling a distribution over policies in a Bayesian deep reinforcement learning setup to propose a transfer strategy. Recent works have shown to induce diversity in the learned policies by maximizing the entropy of a distribution of policies (Bachman et al., 2018; Garnelo et al., 2018) and thus, we postulate that our proposed approach leads to faster exploration resulting in improved transfer learning. We support our hypothesis by demonstrating favorable experimental results on a variety of settings on fully-observable GridWorld and partially observable MiniGrid (Chevalier-Boisvert et al., 2018) environments.

Citations

• Bridging the Gap Between Value and Policy Based Reinforcement Learning
• A unified view of entropy-regularized Markov decision processes
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Related

• Bayesian Transfer Reinforcement Learning with Prior Knowledge Rules
• Adaptive Policy Transfer in Reinforcement Learning
• Transfer RL via the Undo Maps Formalism
• VPE: Variational Policy Embedding for Transfer Reinforcement Learning
• Learn Dynamic-Aware State Embedding for Transfer Learning