LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation

2025/06/02 by Guobin Zhu, Zhu, Guobin, Rui Zhou +5 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Multi-Agent Systems and Negotiation #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2506.01538

openalex publication_date 2025/06/02 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Although Multi-Agent Reinforcement Learning (MARL) is effective for complex multi-robot tasks, it suffers from low sample efficiency and requires iterative manual reward tuning. Large Language Models (LLMs) have shown promise in single-robot settings, but their application in multi-robot systems remains largely unexplored. This paper introduces a novel LLM-Aided MARL (LAMARL) approach, which integrates MARL with LLMs, significantly enhancing sample efficiency without requiring manual design. LAMARL consists of two modules: the first module leverages LLMs to fully automate the generation of prior policy and reward functions. The second module is MARL, which uses the generated functions to guide robot policy training effectively. On a shape assembly benchmark, both simulation and real-world experiments demonstrate the unique advantages of LAMARL. Ablation studies show that the prior policy improves sample efficiency by an average of 185.9% and enhances task completion, while structured prompts based on Chain-of-Thought (CoT) and basic APIs improve LLM output success rates by 28.5%-67.5%. Videos and code are available at https://windylab.github.io/LAMARL/

Citations

• LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments
• To Believe or Not to Believe Your LLM
• Challenges Faced by Large Language Models in Solving Multi-Agent Flocking
• RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Eureka: Human-Level Reward Design via Coding Large Language Models
• Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
• RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
• A Survey of Large Language Models
• Code as Policies: Language Model Programs for Embodied Control
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians
• Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences
• RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
• On Learning Intrinsic Rewards for Policy Gradient Methods
• Bearing Rigidity Theory and its Applications for Control and Estimation of Network Systems: Life Beyond Distance Rigidity
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Count-Based Exploration with Neural Density Models

Cited by

• LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems
• Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs
• Towards Transparent and Incentive-Compatible Collaboration in Decentralized LLM Multi-Agent Systems: A Blockchain-Driven Approach

Related

• MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation
• CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks
• MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models
• MARBLER: An Open Platform for Standardized Evaluation of Multi-Robot Reinforcement Learning Algorithms
• NeuronsMAE: A Novel Multi-Agent Reinforcement Learning Environment for Cooperative and Competitive Multi-Robot Tasks