Attention-Enhanced Reservoir Computing

2023/12/27 by Felix Köster, Kazutaka Kanno, Köster, Felix +5 · 2 citations
Computer Science · Neuroscience · #Emerging Technologies (cs.ET) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.2312.16503

openalex publication_date 2023/12/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Photonic reservoir computing has been successfully utilized in time-series prediction as the need for hardware implementations has increased. Prediction of chaotic time series remains a significant challenge, an area where the conventional reservoir computing framework encounters limitations of prediction accuracy. We introduce an attention mechanism to the reservoir computing model in the output stage. This attention layer is designed to prioritize distinct features and temporal sequences, thereby substantially enhancing the prediction accuracy. Our results show that a photonic reservoir computer enhanced with the attention mechanism exhibits improved prediction capabilities for smaller reservoirs. These advancements highlight the transformative possibilities of reservoir computing for practical applications where accurate prediction of chaotic time series is crucial.

Cited by

• Enhancing Time-Series Anomaly Detection by Integrating Spectral-Residual Bottom-Up Attention with Reservoir Computing
• Reservoir Computing as a Language Model

Related

• Photonic reservoir computing with complex networks
• Attention-Enhanced Reservoir Computing as a Multiple Dynamical System Approximator
• Optical next generation reservoir computing
• Approximate reservoir computing with a semiconductor laser for reducing energy consumption
• Hybrid Photonic-Quantum Reservoir Computing For Time-Series Prediction