Learning to Communicate to Solve Riddles with Deep Distributed Recurrent\n Q-Networks

2016/02/08 by Jakob Foerster, Jakob N. Foerster, Yannis M. Assael +7 · 3 voices · 7 citations
Computer Science · Engineering · #Distributed Control Multi-Agent Systems #Modular Robots and Swarm Intelligence #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1602.02672

openalex publication_date 2016/02/08 · openalex created_date 2022/09/30 · openalex updated_date 2026/07/28

Abstract

We propose deep distributed recurrent Q-networks (DDRQN), which enable teams\nof agents to learn to solve communication-based coordination tasks. In these\ntasks, the agents are not given any pre-designed communication protocol.\nTherefore, in order to successfully communicate, they must first automatically\ndevelop and agree upon their own communication protocol. We present empirical\nresults on two multi-agent learning problems based on well-known riddles,\ndemonstrating that DDRQN can successfully solve such tasks and discover elegant\ncommunication protocols to do so. To our knowledge, this is the first time deep\nreinforcement learning has succeeded in learning communication protocols. In\naddition, we present ablation experiments that confirm that each of the main\ncomponents of the DDRQN architecture are critical to its success.\n

Citations

• Empirically Evaluating Multiagent Learning Algorithms
• Consensus and Cooperation in Networked Multi-Agent Systems
• Dueling Network Architectures for Deep Reinforcement Learning

Cited by

• Emergent Communication in a Multi-Modal, Multi-Step Referential Game
• Adaptive Event-Triggered Policy Gradient for Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
• Mastering Complex Control in MOBA Games with Deep Reinforcement Learning
• Towards Brain-inspired System: Deep Recurrent Reinforcement Learning for Simulated Self-driving Agent
• Cell Selection with Deep Reinforcement Learning in Sparse Mobile Crowdsensing

Discussions

• Learning to Solve Riddles with Deep Distributed Recurrent Q-Networks [pdf] [hn, 44 points, 0 comments]
• Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Nets [hn, 6 points, 0 comments]
• Using deep reinforcement learning to solve interview brain-teasers | http://arxiv.org/pdf/1602.02672v1.pdf https://t.co/0UFUIdPKt0 [bsky, 0 points, 0 comments]

Related

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Learning to Communicate in Multi-Agent Reinforcement Learning : A Review
• FCMNet: Full Communication Memory Net for Team-Level Cooperation in Multi-Agent Systems
• Multiagent Cooperation and Competition with Deep Reinforcement Learning
• Learning Structured Communication for Multi-agent Reinforcement Learning