Unsupervised Video Object Segmentation for Deep Reinforcement Learning

2018/05/20 by Vik Goel, Goel, Vik, Jameson Weng +3 · 1 citation
Computer Science · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1805.07780

openalex publication_date 2018/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a new technique for deep reinforcement learning that automatically detects moving objects and uses the relevant information for action selection. The detection of moving objects is done in an unsupervised way by exploiting structure from motion. Instead of directly learning a policy from raw images, the agent first learns to detect and segment moving objects by exploiting flow information in video sequences. The learned representation is then used to focus the policy of the agent on the moving objects. Over time, the agent identifies which objects are critical for decision making and gradually builds a policy based on relevant moving objects. This approach, which we call Motion-Oriented REinforcement Learning (MOREL), is demonstrated on a suite of Atari games where the ability to detect moving objects reduces the amount of interaction needed with the environment to obtain a good policy. Furthermore, the resulting policy is more interpretable than policies that directly map images to actions or values with a black box neural network. We can gain insight into the policy by inspecting the segmentation and motion of each object detected by the agent. This allows practitioners to confirm whether a policy is making decisions based on sensible information.

Citations

• Playing Atari with Deep Reinforcement Learning
• Gradient-based learning applied to document recognition
• Learning to Navigate in Complex Environments
• SfM-Net: Learning of Structure and Motion from Video
• Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning
• Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Unsupervised Learning of Depth and Ego-Motion from Video
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Cited by

• Combining Pre-Trained Models for Enhanced Feature Representation in Reinforcement Learning

Related

• Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning
• Learning Features by Watching Objects Move
• Object-sensitive Deep Reinforcement Learning
• Unsupervised Learning for Physical Interaction through Video Prediction
• Transferring Agent Behaviors from Videos via Motion GANs