2022/03/08 by Jaleh Zand, Zand, Jaleh, Jack Parker-Holder +3 · 1 citation
Computer Science · Economics, Econometrics and Finance · Social Sciences · #Artificial Intelligence (cs.AI) #Computer Science and Game Theory (cs.GT) #Evolutionary Game Theory and Cooperation #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Sports Analytics and Performance
paper · pdf · doi:10.48550/arxiv.2203.08015
openalex publication_date 2022/03/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Training agents in cooperative settings offers the promise of AI agents able to interact effectively with humans (and other agents) in the real world. Multi-agent reinforcement learning (MARL) has the potential to achieve this goal, demonstrating success in a series of challenging problems. However, whilst these advances are significant, the vast majority of focus has been on the self-play paradigm. This often results in a coordination problem, caused by agents learning to make use of arbitrary conventions when playing with themselves. This means that even the strongest self-play agents may have very low cross-play with other agents, including other initializations of the same algorithm. In this paper we propose to solve this problem by adapting agent strategies on the fly, using a posterior belief over the other agents' strategy. Concretely, we consider the problem of selecting a strategy from a finite set of previously trained agents, to play with an unknown partner. We propose an extension of the classic statistical technique, Gibbs sampling, to update beliefs about other agents and obtain close to optimal ad-hoc performance. Despite its simplicity, our method is able to achieve strong cross-play with unseen partners in the challenging card game of Hanabi, achieving successful ad-hoc coordination without knowledge of the partner's strategy a priori.