The Hanabi Challenge: A New Frontier for AI Research

2019/02/01 by Nolan Bard, Jakob Foerster, Jakob N. Foerster +14 · 4 voices · 238 citations
Computer Science · Decision Sciences · Mathematics · Psychology · #Advanced Bandit Algorithms Research #Artificial Intelligence in Games #Artificial intelligence #Cognitive science #Computer science #Data science #Domain (mathematical analysis) #Frontier #Human–computer interaction #Imperfect #Perfect information #Psychology #Reinforcement Learning in Robotics #State (computer science) #cs.AI #cs.LG #stat.ML

paper · pdf · open access · doi:10.1016/j.artint.2019.103216

published in Artificial Intelligence 280, 103216 (Elsevier BV) · 32 pages, 5 figures, In Press (Artificial Intelligence)

openalex publication_date 2019/11/27 · arxiv created 2019/12/06 · arxiv updated 2019/12/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.

Citations

• DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

Cited by

• Student of Games: A unified learning algorithm for both perfect and imperfect information games
• Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review
• Assistax: A Multi-Agent Hardware-Accelerated Reinforcement Learning Benchmark for Assistive Robotics
• SAGA: Scene-Aware, Goal-Evolving Agents for Long-Horizon Strategy Game Planning
• Human-like autonomy emerges from self-play and a pinch of human data
• Learning to Drive in New Cities Without Human Demonstrations
• Characterizing Language Use in a Collaborative Situated Game
• Adversarially Guided Self-Play for Adopting Social Conventions
• Open Problems in Cooperative AI
• The Pace of Artificial Intelligence Innovations: Speed, Talent, and Trial-and-Error
• Observer, Not Player: Simulating Theory of Mind in LLMs through Game Observation
• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia
• The Curse of Shared Knowledge: Recursive Belief Reasoning in a Coordination Game with Imperfect Information
• High entropy leads to symmetry-equivariant policies in Dec-POMDPs
• PettingZoo: Gym for Multi-Agent Reinforcement Learning
• Robust and Diverse Multi-Agent Learning via Rational Policy Gradient
• Off-Belief Learning
• Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale
• Centralized Model and Exploration Policy for Multi-Agent RL
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs
• OpenSpiel: A Framework for Reinforcement Learning in Games
• Monopoly Deal: A Benchmark Environment for Bounded One-Sided Response Games
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
• HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning
• Learning to Play No-Press Diplomacy with Best Response Policy Iteration
• Knowledge and Common Knowledge of Strategies
• A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning
• Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates
• Evaluating Language Models' Evaluations of Games
• AI Agents for the Dhumbal Card Game: A Comparative Study
• "Other-Play" for Zero-Shot Coordination
• Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi
• Mindful Human Digital Twins: Integrating Theory of Mind with multi-agent reinforcement learning
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• The Heterogeneous Multi-Agent Challenge
• Benchmarking Open-Ended Multi-Agent Coordination in Language Agents
• K-Level Policy Gradients for Multi-Agent Reinforcement Learning
• Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning
• Neural Recursive Belief States in Multi-Agent Reinforcement Learning
• Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems
• The Yokai Learning Environment: Tracking Beliefs Over Space and Time
• Theory of Mind for Deep Reinforcement Learning in Hanabi
• Evolutionary Optimization of Deep Learning Agents for Sparrow Mahjong
• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
• Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical Systems
• Moving Out: Physically-grounded Human-AI Collaboration
• Remembering the Markov Property in Cooperative MARL
• Creative Captioning: An AI Grand Challenge Based on the Dixit Board Game
• Joint Policy Search for Multi-agent Collaboration with Imperfect Information
• Modeling Sensorimotor Coordination as Multi-Agent Reinforcement Learning with Differentiable Communication
• Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning
• CREW-WILDFIRE: Benchmarking Agentic Multi-Agent Collaborations at Scale
• Ella: Embodied Social Agents with Lifelong Memory
• A Survey of Continual Reinforcement Learning
• Ad-Hoc Human-AI Coordination Challenge
• The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
• Compute Trends Across Three Eras of Machine Learning
• Emergent Multi-Agent Communication in the Deep Learning Era
• Neural Replicator Dynamics
• VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments
• Multiplayer Support for the Arcade Learning Environment
• Re-determinizing Information Set Monte Carlo Tree Search in Hanabi
• Biological Pathway Guided Gene Selection Through Collaborative Reinforcement Learning
• Communication-Efficient Desire Alignment for Embodied Agent-Human Adaptation
• Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems
• Artificial Intelligence and Systemic Risk
• GRPO Does Not Close the Multi-Agent Coordination Gap
• TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning
• Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details

Discussions

• The Hanabi Challenge: A New Frontier for AI Research [hn, 150 points, 66 comments]
• DeepMind and Google Brain – The Hanabi Challenge: A New Frontier for AI Research [hn, 4 points, 0 comments]
• Sur la théorie de l'esprit et la formalisation des processus cognitifs impliqués par certains jeux, il y avait eu ce magnifique papier par deepmind sur Hannabi: arxiv.org/abs/1902.00506 [bsky, 2 points, 1 comments]
• The Hanabi Challange: A New Frontier for AI Research [hn, 2 points, 0 comments]

Related

• Theory of Mind for Deep Reinforcement Learning in Hanabi
• LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game
• Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates
• Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi
• Ad-Hoc Human-AI Coordination Challenge