Multiagent Soft Q-Learning

2018/04/25 by Wei, Ermo, Wicke, Drew, Freelan, David +1 · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1804.09817

Abstract

Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic pathology known as relative overgeneralization. To resolve this issue, we propose Multiagent Soft Q-learning, which can be seen as the analogue of applying Q-learning to continuous controls. We compare our method to MADDPG, a state-of-the-art approach, and show that our method achieves better coordination in multiagent cooperative tasks, converging to better local optima in the joint action space.

Cited by

• Multi-level Advantage Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

Related

• Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learning
• Regularized Softmax Deep Multi-Agent Q-Learning
• Multiagent Cooperation and Competition with Deep Reinforcement Learning
• Balancing Two-Player Stochastic Games with Soft Q-Learning
• Mixed Q-Functionals: Advancing Value-Based Methods in Cooperative MARL with Continuous Action Domains