MMD-MIX: Value Function Factorisation with Maximum Mean Discrepancy for Cooperative Multi-Agent Reinforcement Learning

2021/06/22 by Xu, Zhiwei, Li, Dapeng, Bai, Yunpeng +1
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)

paper · doi:10.48550/arxiv.2106.11652

Abstract

In the real world, many tasks require multiple agents to cooperate with each other under the condition of local observations. To solve such problems, many multi-agent reinforcement learning methods based on Centralized Training with Decentralized Execution have been proposed. One representative class of work is value decomposition, which decomposes the global joint Q-value Q_jt into individual Q-values Q_a to guide individuals' behaviors, e.g. VDN (Value-Decomposition Networks) and QMIX. However, these baselines often ignore the randomness in the situation. We propose MMD-MIX, a method that combines distributional reinforcement learning and value decomposition to alleviate the above weaknesses. Besides, to improve data sampling efficiency, we were inspired by REM (Random Ensemble Mixture) which is a robust RL algorithm to explicitly introduce randomness into the MMD-MIX. The experiments demonstrate that MMD-MIX outperforms prior baselines in the StarCraft Multi-Agent Challenge (SMAC) environment.

Related

• QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learning
• MCMARL: Parameterizing Value Function via Mixture of Categorical Distributions for Multi-Agent Reinforcement Learning
• Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning
• A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learning
• UneVEn: Universal Value Exploration for Multi-Agent Reinforcement\n Learning