Causal World Models by Unsupervised Deconfounding of Physical Dynamics

2020/12/28 by Minne Li, Mengyue Yang, Li, Minne +9 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2012.14228

openalex publication_date 2020/12/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The capability of imagining internally with a mental model of the world is vitally important for human cognition. If a machine intelligent agent can learn a world model to create a "dream" environment, it can then internally ask what-if questions -- simulate the alternative futures that haven't been experienced in the past yet -- and make optimal decisions accordingly. Existing world models are established typically by learning spatio-temporal regularities embedded from the past sensory signal without taking into account confounding factors that influence state transition dynamics. As such, they fail to answer the critical counterfactual questions about "what would have happened" if a certain action policy was taken. In this paper, we propose Causal World Models (CWMs) that allow unsupervised modeling of relationships between the intervened observations and the alternative futures by learning an estimator of the latent confounding factors. We empirically evaluate our method and demonstrate its effectiveness in a variety of physical reasoning environments. Specifically, we show reductions in sample complexity for reinforcement learning tasks and improvements in counterfactual physical reasoning.

Citations

• Adam: A Method for Stochastic Optimization
• Doubly Robust Policy Evaluation and Learning
• Estimation of Regression Coefficients When Some Regressors are not Always Observed
• Counterfactual Risk Minimization: Learning from Logged Bandit Feedback
• The central role of the propensity score in observational studies for causal effects
• Opponent Modeling in Deep Reinforcement Learning
• Causal Effect Inference with Deep Latent-Variable Models
• Policy Evaluation and Optimization with Continuous Treatments
• Graph networks as learnable physics engines for inference and control
• Recurrent World Models Facilitate Policy Evolution
• Unsupervised Discovery of Parts, Structure, and Dynamics
• COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Unsupervised Learning of Object Keypoints for Perception and Control
• Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories
• Causality for Machine Learning

Cited by

• CG-World: A Large-Scale World-State Dataset and Protocol for World Models

Related

• Learning Local Causal World Models with State Space Models and Attention
• Learning Implicit Causal World Models from Multi-Agent Demonstrations
• Intrinsically Motivated Learning of Causal World Models
• Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
• Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective