Machine Theory of Mind

2018/02/21 by Neil C. Rabinowitz, Frank Perbet, Rabinowitz, Neil C. +10 · 4 voices · 69 citations
Computer Science · Neuroscience · Psychology · Social Sciences · #Artificial intelligence #Bayesian Modeling and Causal Inference #Cognition #Cognitive science #Computer science #Epistemology #Evolutionary Game Theory and Cooperation #Machine learning #Neuroscience #Process (computing) #Psychology #Reinforcement Learning in Robotics #Reinforcement learning #Simple (philosophy) #Theory of mind #cs.AI

paper · pdf · doi:10.48550/arxiv.1802.07740

published in arXiv (Cornell University), 4218-4227 (Cornell University) · 21 pages, 15 figures

openalex publication_date 2018/02/21 · arxiv created 2018/03/12 · arxiv updated 2018/03/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Theory of mind (ToM; Premack & Woodruff, 1978) broadly refers to humans' ability to represent the mental states of others, including their desires, beliefs, and intentions. We propose to train a machine to build such models too. We design a Theory of Mind neural network -- a ToMnet -- which uses meta-learning to build models of the agents it encounters, from observations of their behaviour alone. Through this process, it acquires a strong prior model for agents' behaviour, as well as the ability to bootstrap to richer predictions about agents' characteristics and mental states using only a small number of behavioural observations. We apply the ToMnet to agents behaving in simple gridworld environments, showing that it learns to model random, algorithmic, and deep reinforcement learning agents from varied populations, and that it passes classic ToM tasks such as the "Sally-Anne" test (Wimmer & Perner, 1983; Baron-Cohen et al., 1985) of recognising that others can hold false beliefs about the world. We argue that this system -- which autonomously learns how to model other agents in its world -- is an important step forward for developing multi-agent AI systems, for building intermediating technology for machine-human interaction, and for advancing the progress on interpretable AI.

Citations

• Generative Adversarial Imitation Learning
• Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017

Cited by

• Align AI to Dynamic Human-AI Workflows
• Empathy as Predictive Misalignment Tolerance: A Co-Regulation Framework and the Regime Structure of Dialogue Repair
• Modeling Others' Minds as Code
• General agents contain world models
• Policy-Conditioned Policies for Multi-Agent Task Solving
• ClinTutor-R1: Advancing Scalable and Robust One-to-Many Alignment in Clinical Socratic Education
• A Computable Game-Theoretic Framework for Multi-Agent Theory of Mind
• Think How Your Teammates Think: Active Inference Can Benefit Decentralized Execution
• Adaptively Coordinating with Novel Partners via Learned Latent Strategies
• Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy
• A Regularized Opponent Model with Maximum Entropy Objective
• Guilty Artificial Minds
• Learning Latent Representations to Influence Multi-Agent Interaction
• CooperBench: Why Coding Agents Cannot be Your Teammates Yet
• Inter-Agent Relative Representations for Multi-Agent Option Discovery
• Human Machine Social Hybrid Intelligence:A Collaborative Decision Making Framework for Large Model Agent Groups and Human Experts
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
• COOPERA: Continual Open-Ended Human-Robot Assistance
• World Models Should Prioritize the Unification of Physical and Social Dynamics
• Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challenges
• X-Ego: Acquiring Team-Level Tactical Situational Awareness via Cross-Egocentric Contrastive Video Representation Learning
• Causal Analysis of Agent Behavior for AI Safety
• Socialized Learning and Emergent Behaviors in Multi-Agent Systems based on Multimodal Large Language Models
• Probabilistic Modeling of Intentions in Socially Intelligent LLM Agents
• A systematic review of algorithm aversion in augmented decision making
• Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations
• Unifying Agent Interaction and World Information for Multi-agent Coordination
• Not Everyone Wins with LLMs: Behavioral Patterns and Pedagogical Implications in AI-assisted Data Analysis
• On the potential for open-endedness in neural networks
• Online Learning of Deceptive Policies under Intermittent Observation
• One Model, Two Minds: A Context-Gated Graph Learner that Recreates Human Biases
• ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions
• Neural Recursive Belief States in Multi-Agent Reinforcement Learning
• Interactive Agent Modeling by Learning to Probe
• Learning to Communicate Implicitly By Actions
• The Yokai Learning Environment: Tracking Beliefs Over Space and Time
• Meta-Learning of Structured Task Distributions in Humans and Machines
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks
• What Do Agents Think One Another Want? Level-2 Inverse Games for Inferring Agents' Estimates of Others' Objectives
• When not to help: planning for lasting human-AI collaboration
• Remembering the Markov Property in Cooperative MARL
• On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference
• DPMT: Dual Process Multi-scale Theory of Mind Framework for Real-time Human-AI Collaboration
• Learning Triadic Belief Dynamics in Nonverbal Communication from Videos
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
• Bayesian Inverse Physics for Neuro-Symbolic Robot Learning
• Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration
• Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning
• Can LLMs Play Ô Ăn Quan Game? A Study of Multi-Step Planning and Decision Making
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning
• Theory of Machine Networks: A Case Study
• The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind
• Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner
• Motion Reasoning for Goal-Based Imitation Learning
• Human-Centered Human-AI Collaboration (HCHAC)
• Communication-Efficient Desire Alignment for Embodied Agent-Human Adaptation
• DEL-ToM: Inference-Time Scaling for Theory-of-Mind Reasoning via Dynamic Epistemic Logic
• Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)
• On Memory Mechanism in Multi-Agent Reinforcement Learning
• Amanuensis: The Programmer's Apprentice
• Automated Meta Prompt Engineering for Alignment with the Theory of Mind
• Strategy-Augmented Planning for Large Language Models via Opponent Exploitation
• MindZero: Learning Online Mental Reasoning With Zero Annotations
• RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments
• Theory of Mind in Large Language Models: Assessment and Enhancement
• Who Responds When the Driver Is Gone? A Framework for Holistic Passenger Intent Understanding
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
• Measurement of LLM's Philosophies of Human Nature

Discussions

• Machine Theory of Mind [hn, 207 points, 32 comments]
• Machine Theory of Mind [hn, 2 points, 0 comments]
• Is this more than the idea of having already learned how to adapt over task space? like few-shot multitask learning: if learn a representation over tasks, you don't have to learn, you just adapt. Her [bsky, 1 points, 1 comments]
• A couple of things worth adding IMO although the impact is a live debate - This ToMNet paper arxiv.org/abs/1802.07740 - Work on Multiagent RL arxiv.org/abs/1911.10635 [bsky, 1 points, 1 comments]

Related

• Silico-centric Theory of Mind
• A Brain-inspired Theory of Collective Mind Model for Efficient Social Cooperation
• Robot Learning Theory of Mind through Self-Observation: Exploiting the Intentions-Beliefs Synergy
• Learning Theory of Mind via Dynamic Traits Attribution
• Towards properly implementing Theory of Mind in AI systems: An account of four misconceptions