Control of Memory, Active Perception, and Action in Minecraft

2016/05/30 by Junhyuk Oh, Valliappa Chockalingam, Oh, Junhyuk +5 · 172 citations
Computer Science · Engineering · #Action (physics) #Active perception #Adaptive Dynamic Programming Control #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #FOS: Computer and information sciences #Generalization #Human–computer interaction #Machine Learning (cs.LG) #Observability #Perception #Programming language #Reinforcement Learning in Robotics #Reinforcement learning #Robot #Robotic Locomotion and Control #Set (abstract data type) #cs.AI #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1605.09128

published in arXiv (Cornell University), 2790-2799 (Cornell University) · ICML 2016

arxiv created 2016/05/30 · openalex publication_date 2016/05/30 · arxiv updated 2016/05/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world). We then use these tasks to systematically compare and contrast existing deep reinforcement learning (DRL) architectures with our new memory-based DRL architectures. These tasks are designed to emphasize, in a controllable manner, issues that pose challenges for RL methods including partial observability (due to first-person visual observations), delayed rewards, high-dimensional visual observations, and the need to use active perception in a correct manner so as to perform well in the tasks. While these tasks are conceptually simple to describe, by virtue of having all of these challenges simultaneously they are difficult for current DRL architectures. Additionally, we evaluate the generalization performance of the architectures on environments not used during training. The experimental results show that our new architectures generalize to unseen environments better than existing DRL architectures.

Citations

• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Trust Region Policy Optimization
• Long Short-Term Memory
• Deep Learning in Neural Networks: An Overview
• Neural Turing Machines
• MazeBase: A Sandbox for Learning from Games

Cited by

• Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidance
• Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation
• PuzzleJAX: A Benchmark for Reasoning and Learning
• When Can Model-Free Reinforcement Learning be Enough for Thinking?
• A Critical Investigation of Deep Reinforcement Learning for Navigation
• CraftAssist: A Framework for Dialogue-enabled Interactive Agents
• Chasing Ghosts: Instruction Following as Bayesian State Tracking
• A2: Extracting Cyclic Switchings from DOB-nets for Rejecting Excessive Disturbances
• Deep Reinforcement Learning from Policy-Dependent Human Feedback
• Optimize Neural Fictitious Self-Play in Regret Minimization Thinking
• ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State Reflection
• Generalizing Skills with Semi-Supervised Reinforcement Learning
• The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
• Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition
• Working Memory Graphs
• Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling
• Deep Reinforcement Learning: An Overview
• Understanding and Controlling Memory in Recurrent Neural Networks
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• pix2pockets: Shot Suggestions in 8-Ball Pool from a Single Image in the Wild

Related

• Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds
• JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning
• MineRL: A Large-Scale Dataset of Minecraft Demonstrations
• Sample Efficient Reinforcement Learning through Learning from Demonstrations in Minecraft
• Improving Deep Reinforcement Learning in Minecraft with Action Advice