A Novel Update Mechanism for Q-Networks Based On Extreme Learning\n Machines

2020/06/04 by Callum Wilson, Annalisa Riccardi, Wilson, Callum +3 · 1 citation
Computer Science · Engineering · #Active learning (machine learning) #Adaptive Dynamic Programming Control #Advanced Memory and Neural Computing #Artificial intelligence #Artificial neural network #Benchmark (surveying) #Computer science #Engineering #Extreme learning machine #FOS: Computer and information sciences #Function (biology) #Instance-based learning #Learning classifier system #Machine Learning (cs.LG) #Machine Learning and ELM #Machine learning #Mechanism (biology) #Neural and Evolutionary Computing (cs.NE) #Online machine learning #Q-learning #Reinforcement learning #Stochastic Gradient Optimization Techniques #Task (project management) #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.2006.02986

published in arXiv (Cornell University) (Cornell University) · Accepted for IJCNN/WCCI 2020

arxiv created 2020/06/04 · openalex publication_date 2020/06/04 · arxiv updated 2020/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08

Abstract

Reinforcement learning is a popular machine learning paradigm which can find\nnear optimal solutions to complex problems. Most often, these procedures\ninvolve function approximation using neural networks with gradient based\nupdates to optimise weights for the problem being considered. While this common\napproach generally works well, there are other update mechanisms which are\nlargely unexplored in reinforcement learning. One such mechanism is Extreme\nLearning Machines. These were initially proposed to drastically improve the\ntraining speed of neural networks and have since seen many applications. Here\nwe attempt to apply extreme learning machines to a reinforcement learning\nproblem in the same manner as gradient based updates. This new algorithm is\ncalled Extreme Q-Learning Machine (EQLM). We compare its performance to a\ntypical Q-Network on the cart-pole task - a benchmark reinforcement learning\nproblem - and show EQLM has similar long-term learning performance to a\nQ-Network.\n

Citations

• Playing Atari with Deep Reinforcement Learning
• Deep Reinforcement Learning that Matters

Cited by

• High-order Regularization for Machine Learning and Learning-based Control

Related

• Deep Q-Networks for Accelerating the Training of Deep Neural Networks
• Using Deep Q-Learning to Control Optimization Hyperparameters
• Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning
• Deep Q-Learning with Gradient Target Tracking
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning