Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systems

2025/01/11 by Hu, Liyuan
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)

paper · doi:10.48550/arxiv.2501.06554

Abstract

This paper presents a hierarchical reinforcement learning (RL) approach to address the agent grouping or pairing problem in cooperative multi-agent systems. The goal is to simultaneously learn the optimal grouping and agent policy. By employing a hierarchical RL framework, we distinguish between high-level decisions of grouping and low-level agents' actions. Our approach utilizes the CTDE (Centralized Training with Decentralized Execution) paradigm, ensuring efficient learning and scalable execution. We incorporate permutation-invariant neural networks to handle the homogeneity and cooperation among agents, enabling effective coordination. The option-critic algorithm is adapted to manage the hierarchical decision-making process, allowing for dynamic and optimal policy adjustments.

Related

• Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation
• HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism
• Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning
• Structured Diversification Emergence via Reinforced Organization Control and Hierarchical Consensus Learning
• Learning to Collaborate by Grouping: a Consensus-oriented Strategy for Multi-agent Reinforcement Learning