Generalizable Agent Modeling for Agent Collaboration-Competition Adaptation with Multi-Retrieval and Dynamic Generation

2025/06/20 by Wang, Chenxu, Jin, Yonggang, Hu, Cheng +7
Computer Science · #Adaptation (eye) #Artificial Intelligence (cs.AI) #Benchmark (surveying) #Class (philosophy) #Code (set theory) #Domain Adaptation and Few-Shot Learning #Encoder #FOS: Computer and information sciences #Intelligent agent #Mobile Crowdsensing and Crowdsourcing #Multi-agent system #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics #Reinforcement learning #Task (project management)

paper · pdf · doi:10.48550/arxiv.2506.16718

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Adapting a single agent to a new multi-agent system brings challenges, necessitating adjustments across various tasks, environments, and interactions with unknown teammates and opponents. Addressing this challenge is highly complex, and researchers have proposed two simplified scenarios, Multi-agent reinforcement learning for zero-shot learning and Ad-Hoc Teamwork. Building on these foundations, we propose a more comprehensive setting, Agent Collaborative-Competitive Adaptation (ACCA), which evaluates an agent to generalize across diverse scenarios, tasks, and interactions with both unfamiliar opponents and teammates. In ACCA, agents adjust to task and environmental changes, collaborate with unseen teammates, and compete against unknown opponents. We introduce a new modeling approach, Multi-Retrieval and Dynamic Generation (MRDG), that effectively models both teammates and opponents using their behavioral trajectories. This method incorporates a positional encoder for varying team sizes and a hypernetwork module to boost agents' learning and adaptive capabilities. Additionally, a viewpoint alignment module harmonizes the observational perspectives of retrieved teammates and opponents with the learning agent. Extensive tests in benchmark scenarios like SMAC, Overcooked-AI, and Melting Pot show that MRDG significantly improves robust collaboration and competition with unseen teammates and opponents, surpassing established baselines. Our code is available at: https://github.com/vcis-wangchenxu/MRDG.git

Citations

• Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization
• Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration
• METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
• TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play
• RPM: Generalizable Behaviors for Multi-Agent Reinforcement Learning
• Quantum Multi-Agent Meta Reinforcement Learning
• Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution
• DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization
• Large-Scale Retrieval for Reinforcement Learning
• A Generalist Agent
• A Survey of Ad Hoc Teamwork Research
• Task Aligned Generative Meta-learning for Zero-shot Learning
• Open World Compositional Zero-Shot Learning
• Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
• Distributed Multi-agent Meta Learning for Trajectory Design in Wireless Drone Networks
• Generalized Zero-Shot Learning Via Over-Complete Distribution
• An Efficient Transfer Learning Framework for Multiagent Reinforcement Learning
• Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
• On the Utility of Learning about Humans for Human-AI Coordination
• Diverse Agents for Ad-Hoc Cooperation in Hanabi
• The StarCraft Multi-Agent Challenge
• Harnessing GANs for Zero-shot Learning of New Classes in Visual Speech Recognition
• Addressing Function Approximation Error in Actor-Critic Methods
• Diversity is All You Need: Learning Skills without a Reward Function
• Attention Is All You Need
• HyperNetworks

Related

• Multi-Agent Interplay in a Competitive Survival Environment
• Multi-Task Multi-Agent Shared Layers are Universal Cognition of Multi-Agent Coordination
• Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration
• Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments
• N-Agent Ad Hoc Teamwork