Learning Transition Models with Time-delayed Causal Relations

2020/08/04 by Junchi Liang, Abdeslam Boularias, Liang, Junchi +1
Computer Science · Engineering · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2008.01593

openalex publication_date 2020/08/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces an algorithm for discovering implicit and delayed causal relations between events observed by a robot at arbitrary times, with the objective of improving data-efficiency and interpretability of model-based reinforcement learning (RL) techniques. The proposed algorithm initially predicts observations with the Markov assumption, and incrementally introduces new hidden variables to explain and reduce the stochasticity of the observations. The hidden variables are memory units that keep track of pertinent past events. Such events are systematically identified by their information gains. The learned transition and reward models are then used for planning. Experiments on simulated and real robotic tasks show that this method significantly improves over current RL techniques.

Citations

• Show, Attend and Tell: Neural Image Caption Generation with Visual\n Attention
• Playing Atari with Deep Reinforcement Learning
• Episodic Exploration for Deep Deterministic Policies: An Application to\n StarCraft Micromanagement Tasks
• Interaction Networks for Learning about Objects, Relations and Physics
• Recurrent Environment Simulators
• Learning Longer-term Dependencies in RNNs with Auxiliary Losses

Related

• Causality-Based Reinforcement Learning Method for Multi-Stage Robotic Tasks
• Causal Reinforcement Learning for Optimisation of Robot Dynamics in Unknown Environments
• Learning Causal Structure Distributions for Robust Planning
• Working Paper: Active Causal Structure Learning with Latent Variables: Towards Learning to Detour in Autonomous Robots
• From Continual Learning to Causal Discovery in Robotics