A fast memoryless predictive algorithm in a chain of recurrent neural networks

2020/10/05 by Boris Rubinstein, Rubinstein, Boris
Biochemistry, Genetics and Molecular Biology · Computer Science · #37N30 #68T07 #Dynamical Systems (math.DS) #FOS: Mathematics #Machine Learning and Algorithms #Machine Learning in Bioinformatics #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2010.02115

openalex publication_date 2020/10/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in natural neural systems. When the network is used recursively to predict sequence of values the proposed algorithm does not require to store the original input sequence. It increases robustness of the new approach compared to the standard moving/expanding window predictive procedure. We consider requirements on trained networks that allow to implement the proposed algorithm and discuss them in the neuroscience context.

Citations

• Long Short-Term Memory
• Language Models are Few-Shot Learners

Related

• On a novel training algorithm for sequence-to-sequence predictive recurrent networks
• A fast noise filtering algorithm for time series prediction using recurrent neural networks
• It's a super deal -- train recurrent network on noisy data and get smooth prediction free
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• Alleviate Exposure Bias in Sequence Prediction with Recurrent Neural Networks