A Deep Hierarchical Approach to Lifelong Learning in Minecraft

2016/04/25 by Chen Tessler, Tessler, Chen, Shahar Givony +7 · 1 voice · 11 citations
Computer Science · Decision Sciences · #Adaptive Dynamic Programming Control #Advanced Bandit Algorithms Research #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1604.07255

openalex publication_date 2016/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a lifelong learning system that has the ability to reuse and transfer knowledge from one task to another while efficiently retaining the previously learned knowledge-base. Knowledge is transferred by learning reusable skills to solve tasks in Minecraft, a popular video game which is an unsolved and high-dimensional lifelong learning problem. These reusable skills, which we refer to as Deep Skill Networks, are then incorporated into our novel Hierarchical Deep Reinforcement Learning Network (H-DRLN) architecture using two techniques: (1) a deep skill array and (2) skill distillation, our novel variation of policy distillation (Rusu et. al. 2015) for learning skills. Skill distillation enables the HDRLN to efficiently retain knowledge and therefore scale in lifelong learning, by accumulating knowledge and encapsulating multiple reusable skills into a single distilled network. The H-DRLN exhibits superior performance and lower learning sample complexity compared to the regular Deep Q Network (Mnih et. al. 2015) in sub-domains of Minecraft.

Citations

• Hierarchical Solution of Markov Decision Processes using Macro-actions
• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
• Dueling Network Architectures for Deep Reinforcement Learning
• Prioritized Experience Replay
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Strategic Attentive Writer for Learning Macro-Actions

Cited by

• Learning to Drive Safely with Hybrid Options
• Learning to Ball: Composing Policies for Long-Horizon Basketball Moves
• Robust Reinforcement Learning for Continuous Control with Model Misspecification
• Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
• Visualizing Dynamics: from t-SNE to SEMI-MDPs
• Deep Learning in Mobile and Wireless Networking: A Survey
• AI Research Considerations for Human Existential Safety (ARCHES)
• Representative Task Self-selection for Flexible Clustered Lifelong Learning
• Lifelong Learning using Eigentasks: Task Separation, Skill Acquisition, and Selective Transfer
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Some Considerations on Learning to Explore via Meta-Reinforcement Learning

Discussions

• A Deep Hierarchical Approach to Lifelong Learning in Minecraft [hn, 46 points, 7 comments]

Related

• Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft
• Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks
• JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning
• Lifelong Learning with Dynamically Expandable Networks
• Control of Memory, Active Perception, and Action in Minecraft