Learning long-term dependencies with gradient descent is difficult

1994/03/01 by Y. Bengio, P. Simard, P. Frasconi · 52 citations

paper · doi:10.1109/72.279181

Cited by

• Reactive Transformer (RxT) -- Stateful Real-Time Processing for Event-Driven Reactive Language Models
• HRM-Text: Efficient Pretraining Beyond Scaling
• Human-AI Collaborative Uncertainty Quantification
• Dopamine-driven synaptic credit assignment in neural networks
• Extreme value forecasting using relevance-based data augmentation with deep learning models
• Weights initialization of neural networks for function approximation
• Artificial Hippocampus Networks for Efficient Long-Context Modeling
• HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
• Gather-Scatter Mamba: Accelerating Propagation with Efficient State Space Model
• Identifying Information-Transfer Nodes in a Recurrent Neural Network Reveals Dynamic Representations
• Room Impulse Response Prediction with Neural Networks: From Energy Decay Curves to Perceptual Validation
• Deep Learning-Based Prediction of Energy Decay Curves from Room Geometry and Material Properties
• Who invented deep residual learning?
• PALQO: Physics-informed Model for Accelerating Large-scale Quantum Optimization
• Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications
• The Topological Trouble With Transformers
• Neural Network Based Framework for Passive Intermodulation Cancellation in MIMO Systems
• Deep Learning Inductive Biases for fMRI Time Series Classification during Resting-state and Movie-watching
• M4GN: Mesh-based Multi-segment Hierarchical Graph Network for Dynamic Simulations
• AquaCast: Urban Water Dynamics Forecasting with Precipitation-Informed Multi-Input Transformer
• Measuring General Associations in Time Series: An Adaptation and Empirical Evaluation of the CODEC Coefficient in Determining Autoregressive Dynamics
• Uncertainty-Aware Neural Networks for Fuzzy Dark Matter Model Selection from \texorpdfstringx_{\rm HI}x_HI Measurements
• Deep Learning for Conditional Asset Pricing Models
• A Brain-Inspired Gating Mechanism Unlocks Robust Computation in Spiking Neural Networks
• Motor Imagery EEG Signal Classification Using Minimally Random Convolutional Kernel Transform and Hybrid Deep Learning
• A constrained optimization approach to nonlinear system identification through simulation error minimization
• Deep Residual Echo State Networks: exploring residual orthogonal connections in untrained Recurrent Neural Networks
• C¹-approximation with rational functions and rational neural networks
• Revisiting associative recall in modern recurrent models
• Frozen in Time: Parameter-Efficient Time Series Transformers via Reservoir-Induced Feature Expansion and Fixed Random Dynamics
• Wormhole Dynamics in Deep Neural Networks
• Autoregressive Typical Thermal States
• Bi-Axial Transformers: Addressing the Increasing Complexity of EHR Classification
• The Role of Entanglement in Quantum Reservoir Computing with Coupled Kerr Nonlinear Oscillators
• Lameness detection in dairy cows using pose estimation and bidirectional LSTMs
• Reinforcement learning in densely recurrent biological networks
• Chi-Geometry: A Library for Benchmarking Chirality Prediction of GNNs
• Neural Networks with Orthogonal Jacobian
• The Vanishing Gradient Problem for Stiff Neural Differential Equations
• Forecasting NCAA Basketball Outcomes with Deep Learning: A Comparative Study of LSTM and Transformer Models
• Hebbian Memory-Augmented Recurrent Networks: Engram Neurons in Deep Learning
• Spatial-Temporal Reinforcement Learning for Network Routing with Non-Markovian Traffic
• Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments
• Quantum Boltzmann Machines using Parallel Annealing for Medical Image Classification
• Physics-guided gated recurrent units for inversion-based feedforward control
• Global weather forecasting through deep learning: applications and challenges
• A Wavelength-Aware Unsupervised Learning Approach for Large, Multicolor, Photometric Surveys
• The Serial Scaling Hypothesis
• Data-Driven Dimensional Synthesis of Diverse Planar Four-bar Function Generation Mechanisms via Direct Parameterization
• Data-Driven Reconstruction and Characterization of Stochastic Dynamics via Dynamical Mode Decomposition
• Introduction to Quantum Error Correction with Stabilizer Codes
• RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling