My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013

2013/12/19 by Jürgen Schmidhuber, Schmidhuber, Jürgen
Computer Science · #Anomaly Detection Techniques and Applications #Computational Physics and Python Applications #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE) #Time Series Analysis and Forecasting #cs.NE

paper · pdf · doi:10.48550/arxiv.1312.5548

11 pages. As a machine learning researcher I am obsessed with proper credit assignment. This draft is the result of an experiment in rapid massive open online peer review. Since 20 September 2013, subsequent revisions published under http://www.deeplearning.me have absorbed many suggestions for improvements by experts

arxiv created 2013/12/19 · openalex publication_date 2013/12/19 · arxiv updated 2013/12/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep Learning has attracted significant attention in recent years. Here I present a brief overview of my first Deep Learner of 1991, and its historic context, with a timeline of Deep Learning highlights.

Citations

• Applied Optimal Control: Optimization, Estimation, and Control
• Natural Gradient Works Efficiently in Learning
• Long Short-Term Memory
• Gradient-based learning applied to document recognition
• A Novel Connectionist System for Unconstrained Handwriting Recognition
• Learning to Forget: Continual Prediction with LSTM
• Backpropagation Applied to Handwritten Zip Code Recognition
• ImageNet classification with deep convolutional neural networks

Related

• Deep Learning: Our Miraculous Year 1990-1991
• Deep Learning: A Critical Appraisal
• Deep Neural Networks - A Brief History
• A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas
• On the Origin of Deep Learning