2019/04/30 by Kacper Kielak, Kielak, Kacper
Computer Science · #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.1904.13255
openalex publication_date 2019/04/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Reinforcement learning has seen great advancements in the past five years.\nThe successful introduction of deep learning in place of more traditional\nmethods allowed reinforcement learning to scale to very complex domains\nachieving super-human performance in environments like the game of Go or\nnumerous video games. Despite great successes in multiple domains, these new\nmethods suffer from their own issues that make them often inapplicable to the\nreal world problems. Extreme lack of data efficiency, together with huge\nvariance and difficulty in enforcing safety constraints, is one of the three\nmost prominent issues in the field. Usually, millions of data points sampled\nfrom the environment are necessary for these algorithms to converge to\nacceptable policies.\n This thesis proposes novel Generative Adversarial Imaginative Reinforcement\nLearning algorithm. It takes advantage of the recent introduction of highly\neffective generative adversarial models, and Markov property that underpins\nreinforcement learning setting, to model dynamics of the real environment\nwithin the internal imagination module. Rollouts from the imagination are then\nused to artificially simulate the real environment in a standard reinforcement\nlearning process to avoid, often expensive and dangerous, trial and error in\nthe real environment. Experimental results show that the proposed algorithm\nmore economically utilises experience from the real environment than the\ncurrent state-of-the-art Rainbow DQN algorithm, and thus makes an important\nstep towards sample efficient deep reinforcement learning.\n