Reliable and Fast Recurrent Neural Network Architecture Optimization

2021/06/29 by Andrés Camero, Camero, Andrés, Jamal Toutouh +3
Computer Science · Engineering · #Energy Load and Power Forecasting #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.2106.15295

arxiv created 2021/06/29 · openalex publication_date 2021/06/29 · arxiv updated 2021/06/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

This article introduces Random Error Sampling-based Neuroevolution (RESN), a novel automatic method to optimize recurrent neural network architectures. RESN combines an evolutionary algorithm with a training-free evaluation approach. The results show that RESN achieves state-of-the-art error performance while reducing by half the computational time.

Citations

• Adam: A Method for Stochastic Optimization
• On the difficulty of training Recurrent Neural Networks
• Long Short-Term Memory
• Low-Cost Recurrent Neural Network Expected Performance Evaluation

Related

• Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilation
• An Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution
• Neuroevolution of Neural Network Architectures Using CoDeepNEAT and Keras
• NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications
• NeuroLGP-SM: A Surrogate-assisted Neuroevolution Approach using Linear Genetic Programming