Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learning

2022/10/31 by Jennifer She, She, Jennifer, Jayesh K. Gupta +3 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2210.17540

openalex publication_date 2022/10/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Sparse and delayed rewards pose a challenge to single agent reinforcement learning. This challenge is amplified in multi-agent reinforcement learning (MARL) where credit assignment of these rewards needs to happen not only across time, but also across agents. We propose Agent-Time Attention (ATA), a neural network model with auxiliary losses for redistributing sparse and delayed rewards in collaborative MARL. We provide a simple example that demonstrates how providing agents with their own local redistributed rewards and shared global redistributed rewards motivate different policies. We extend several MiniGrid environments, specifically MultiRoom and DoorKey, to the multi-agent sparse delayed rewards setting. We demonstrate that ATA outperforms various baselines on many instances of these environments. Source code of the experiments is available at https://github.com/jshe/agent-time-attention.

Cited by

• Preference-Guided Learning for Sparse-Reward Multi-Agent Reinforcement Learning

Related

• Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learning
• Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learning
• Multi-Agent Reinforcement Learning with Reward Delays
• Redistributing Rewards Across Time and Agents for Multi-Agent Reinforcement Learning
• Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning