Learning Over Long Time Lags

2016/02/13 by Hojjat Salehinejad, Salehinejad, Hojjat
Computer Science · #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Neural and Evolutionary Computing (cs.NE) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1602.04335

openalex publication_date 2016/02/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The advantage of recurrent neural networks (RNNs) in learning dependencies between time-series data has distinguished RNNs from other deep learning models. Recently, many advances are proposed in this emerging field. However, there is a lack of comprehensive review on memory models in RNNs in the literature. This paper provides a fundamental review on RNNs and long short term memory (LSTM) model. Then, provides a surveys of recent advances in different memory enhancements and learning techniques for capturing long term dependencies in RNNs.

Citations

• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
• Finding Structure in Time
• Bidirectional recurrent neural networks

Related

• Do RNN and LSTM have Long Memory?
• Recent Advances in Recurrent Neural Networks
• A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
• Recurrent Neural Networks and Long Short-Term Memory Networks: Tutorial and Survey
• Residual Memory Networks: Feed-forward approach to learn long temporal dependencies