Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem

2025/07/18 by Turan Orujlu, Christian Gumbsch, Orujlu, Turan +5 · 1 citation
Computer Science · Neuroscience · Psychology · #Child and Animal Learning Development #Embodied and Extended Cognition #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2507.13920

openalex publication_date 2025/07/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Most neural models of causality assume static causal graphs, failing to capture the dynamic and sparse nature of physical interactions where causal relationships emerge and dissolve over time. We introduce the Causal Process Framework and its neural implementation, Causal Process Models (CPMs), for learning sparse, time-varying causal graphs from visual observations. Unlike traditional approaches that maintain dense connectivity, our model explicitly constructs causal edges only when objects actively interact, dramatically improving both interpretability and computational efficiency. We achieve this by casting dynamic interaction-graph construction for world modeling as a multi-agent reinforcement learning problem, where specialized agents sequentially decide which objects are causally connected at each timestep. Our key innovation is a structured representation that factorizes object and force vectors along three learned dimensions (mutability, causal relevance, and control relevance), enabling the automatic discovery of semantically meaningful encodings. We demonstrate that a CPM significantly outperforms dense graph baselines on physical prediction tasks, particularly for longer horizons and varying object counts.

Citations

• Causality in the human niche: lessons for machine learning
• Do Large Language Models Reason Causally Like Us? Even Better?
• Contextualizing predictive minds
• Systems with Switching Causal Relations: A Meta-Causal Perspective
• Reinforcement Learning for Causal Discovery without Acyclicity Constraints
• Transformers need glasses! Information over-squashing in language tasks
• Dynamic Structural Causal Models
• Causal Action Influence Aware Counterfactual Data Augmentation
• How Transformers Learn Causal Structure with Gradient Descent
• CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning
• Causal Interpretation of Self-Attention in Pre-Trained Transformers
• Locality-Aware Graph-Rewiring in GNNs
• How does over-squashing affect the power of GNNs?
• On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology
• Causal Dynamics Learning for Task-Independent State Abstraction
• Understanding over-squashing and bottlenecks on graphs via curvature
• Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domains
• Relating Graph Neural Networks to Structural Causal Models
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
• The Causal-Neural Connection: Expressiveness, Learnability, and Inference
• Causal Influence Detection for Improving Efficiency in Reinforcement Learning
• Toward Causal Representation Learning
• Counterfactual Data Augmentation using Locally Factored Dynamics
• On the Bottleneck of Graph Neural Networks and its Practical Implications
• Contrastive Learning of Structured World Models
• Recurrent Independent Mechanisms
• Causal Discovery with Reinforcement Learning
• How Powerful are Graph Neural Networks?
• Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search
• Relational inductive biases, deep learning, and graph networks
• Learning Sparse Neural Networks through L₀ Regularization
• Graph Attention Networks
• Attention Is All You Need
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Semi-Supervised Classification with Graph Convolutional Networks
• Acceleration of Stochastic Approximation by Averaging
• The Graph Neural Network Model

Cited by

• Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks

Related

• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
• Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications
• Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy
• Explainable Reinforcement Learning Through a Causal Lens
• Disentangled Representations for Causal Cognition