2019/01/27 by Andrew Melnik, Sascha Fleer, Melnik, Andrew +5
Computer Science · Mathematics · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Digital Games and Media #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #cs.AI #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1901.09895
arxiv created 2019/01/27 · openalex publication_date 2019/01/27 · arxiv updated 2019/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on mastering different types of interactions in a broad range of environments. The modules utilize regression, supervised learning, and reinforcement learning algorithms. Results of this case study in different Atari games suggest that human-level performance can be achieved by a learning agent within a human amount of game experience (10-15 minutes game time) when a proper decomposition of an environment or a task is provided. However, automatization of such decomposition remains a challenging problem. This case study shows how a model of a causal structure underlying an environment or a task can benefit learning time and generalization capability of the agent, and argues in favor of exploiting modular structure in contrast to using pure end-to-end learning approaches.