Separation of Concerns in Reinforcement Learning

2016/12/15 by Harm van Seijen, Mehdi Fatemi, van Seijen, Harm +5 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1612.05159

openalex publication_date 2016/12/15 · arxiv created 2017/03/28 · arxiv updated 2017/03/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose a framework for solving a single-agent task by using multiple agents, each focusing on different aspects of the task. This approach has two main advantages: 1) it allows for training specialized agents on different parts of the task, and 2) it provides a new way to transfer knowledge, by transferring trained agents. Our framework generalizes the traditional hierarchical decomposition, in which, at any moment in time, a single agent has control until it has solved its particular subtask. We illustrate our framework with empirical experiments on two domains.

Citations

• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Learning Purposeful Behaviour in the Absence of Rewards
• Learning values across many orders of magnitude

Cited by

• On mechanisms for transfer using landmark value functions in multi-task\n lifelong reinforcement learning

Related

• Efficient Training in Multi-Agent Reinforcement Learning: A Communication-Free Framework for the Box-Pushing Problem
• Strategy-Following Multi-Agent Deep Reinforcement Learning Considering Control Strategies Provided to Other Agents
• Policy Distillation and Value Matching in Multiagent Reinforcement Learning
• Learning Complex Teamwork Tasks Using a Given Sub-task Decomposition
• Learning Symbolic Task Decompositions for Multi-Agent Teams