Deep learning: an overview and main paradigms

2017/01/01 by V. A. Golovko, Vladimir Golovko · 1 citation
Computer Science · Physics and Astronomy · #Generative Adversarial Networks and Image Synthesis #Model Reduction and Neural Networks #Neural Networks and Applications

paper · doi:10.3103/s1060992x16040081

openalex publication_date 2017/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/22

Abstract

In the present paper, we examine and analyze main paradigms of learning of multilayer neural networks starting with a single layer perceptron and ending with deep neural networks, which are considered regarded as a breakthrough in the field of the intelligent data processing. The baselessness of some ideas about the capacity of multilayer neural networks is shown and transition to deep neural networks is justified. We discuss the principal learning models of deep neural networks based on the restricted Boltzmann machine (RBM), an autoassociative approach and a stochastic gradient method with a Rectified Linear Unit (ReLU) activation function of neural elements.

Citations

• Training Products of Experts by Minimizing Contrastive Divergence

Cited by

• Neural networks with image recognition by pairs

Related

• A Tutorial on Deep Neural Networks for Intelligent Systems
• A Selective Overview of Deep Learning
• Deep learning: Technical introduction
• Artificial Neural Network and Deep Learning: Fundamentals and Theory
• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches