Reliably Re-Acting to Partner's Actions with the Social Intrinsic Motivation of Transfer Empowerment

2022/03/07 by Tessa van der Heiden, van der Heiden, Tessa, Herke van Hoof +5
Social Sciences · #Artificial Intelligence (cs.AI) #Evolutionary Game Theory and Cooperation #Experimental Behavioral Economics Studies #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)

paper · pdf · doi:10.48550/arxiv.2203.03355

openalex publication_date 2022/03/07 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28

Abstract

We consider multi-agent reinforcement learning (MARL) for cooperative communication and coordination tasks. MARL agents can be brittle because they can overfit their training partners' policies. This overfitting can produce agents that adopt policies that act under the expectation that other agents will act in a certain way rather than react to their actions. Our objective is to bias the learning process towards finding reactive strategies towards other agents' behaviors. Our method, transfer empowerment, measures the potential influence between agents' actions. Results from three simulated cooperation scenarios support our hypothesis that transfer empowerment improves MARL performance. We discuss how transfer empowerment could be a useful principle to guide multi-agent coordination by ensuring reactiveness to one's partner.

Related

• Robust Multi-Agent Reinforcement Learning with Social Empowerment for Coordination and Communication
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
• From Centralized to Self-Supervised: Pursuing Realistic Multi-Agent Reinforcement Learning
• Strategy-Following Multi-Agent Deep Reinforcement Learning Considering Control Strategies Provided to Other Agents
• Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learning