Reinforcement Learning using Augmented Neural Networks

2018/06/20 by J.D. Shannon, Jack Shannon, Shannon, Jack +3
Computer Science · Engineering · Mathematics · #Advanced Memory and Neural Computing #Neural Networks and Applications #Neural Networks and Reservoir Computing #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1806.07692

7 pages; two columns; 4 figures

arxiv created 2018/06/20 · arxiv updated 2018/06/21

Abstract

Neural networks allow Q-learning reinforcement learning agents such as deep Q-networks (DQN) to approximate complex mappings from state spaces to value functions. However, this also brings drawbacks when compared to other function approximators such as tile coding or their generalisations, radial basis functions (RBF) because they introduce instability due to the side effect of globalised updates present in neural networks. This instability does not even vanish in neural networks that do not have any hidden layers. In this paper, we show that simple modifications to the structure of the neural network can improve stability of DQN learning when a multi-layer perceptron is used for function approximation.

Related

• Does DQN Learn?
• How to Discount Deep Reinforcement Learning: Towards New Dynamic Strategies
• Going Beyond Linear RL: Sample Efficient Neural Function Approximation
• Convergent and Efficient Deep Q Network Algorithm
• Implementing the Deep Q-Network