Learning Action-Transferable Policy with Action Embedding

2019/09/05 by Yu Chen, Yingfeng Chen, Chen, Yu +11
Computer Science · #Action (physics) #Adversarial Robustness in Machine Learning #Algorithm #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer science #Embedding #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Feature (linguistics) #Machine Learning (cs.LG) #Machine learning #Reinforcement Learning in Robotics #Reinforcement learning #Sample (material) #Semantics (computer science) #Space (punctuation) #State (computer science) #Transfer of learning #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1909.02291

openalex publication_date 2019/09/05 · arxiv created 2021/05/10 · arxiv updated 2021/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Transfer learning (TL) is a promising way to improve the sample efficiency of reinforcement learning. However, how to efficiently transfer knowledge across tasks with different state-action spaces is investigated at an early stage. Most previous studies only addressed the inconsistency across different state spaces by learning a common feature space, without considering that similar actions in different action spaces of related tasks share similar semantics. In this paper, we propose a method to learning action embeddings by leveraging this idea, and a framework that learns both state embeddings and action embeddings to transfer policy across tasks with different state and action spaces. Our experimental results on various tasks show that the proposed method can not only learn informative action embeddings but accelerate policy learning.

Citations

• Long Short-Term Memory
• Human-level control through deep reinforcement learning
• Mastering the game of Go with deep neural networks and tree search
• Mutual Alignment Transfer Learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Universal Successor Representations for Transfer Reinforcement Learning
• Knowledge Flow: Improve Upon Your Teachers
• The Natural Language of Actions
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Transfer in Deep Reinforcement Learning Using Successor Features and\n Generalised Policy Improvement
• Neural Network Surgery with Sets
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency

Related

• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer
• Inapplicable Actions Learning for Knowledge Transfer in Reinforcement Learning
• Learn Dynamic-Aware State Embedding for Transfer Learning
• Grounding Language for Transfer in Deep Reinforcement Learning
• Learning Action Representations for Reinforcement Learning