Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning

2025/02/06 by Muhan Lin, Lin, Muhan, Shuyang Shi +17 · 4 citations
Business, Management and Accounting · Computer Science · #Digital Platforms and Economics #FOS: Computer and information sciences #Multiagent Systems (cs.MA) #Open Source Software Innovations

paper · pdf · doi:10.48550/arxiv.2502.03723

openalex publication_date 2025/02/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Credit assignment, the process of attributing credit or blame to individual agents for their contributions to a team's success or failure, remains a fundamental challenge in multi-agent reinforcement learning (MARL), particularly in environments with sparse rewards. Commonly-used approaches such as value decomposition often lead to suboptimal policies in these settings, and designing dense reward functions that align with human intuition can be complex and labor-intensive. In this work, we propose a novel framework where a large language model (LLM) generates dense, agent-specific rewards based on a natural language description of the task and the overall team goal. By learning a potential-based reward function over multiple queries, our method reduces the impact of ranking errors while allowing the LLM to evaluate each agent's contribution to the overall task. Through extensive experiments, we demonstrate that our approach achieves faster convergence and higher policy returns compared to state-of-the-art MARL baselines.

Cited by

• Who Gets the Reward, Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents
• Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs
• Multi-Stakeholder Alignment in LLM-Powered Collaborative AI Systems: A Multi-Agent Framework for Intelligent Tutoring
• Agent²: An Agent-Generates-Agent Framework for Reinforcement Learning Automation

Related

• Redistributing Rewards Across Time and Agents for Multi-Agent Reinforcement Learning
• Who Gets the Reward, Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents
• Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignment
• QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?
• Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning