A SOM-based Gradient-Free Deep Learning Method with Convergence Analysis

2021/01/12 by Shaosheng Xu, Xu, Shaosheng, Jinde Cao +5
Computer Science · Engineering · #Advanced Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2101.05612

openalex publication_date 2021/01/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As gradient descent method in deep learning causes a series of questions, this paper proposes a novel gradient-free deep learning structure. By adding a new module into traditional Self-Organizing Map and introducing residual into the map, a Deep Valued Self-Organizing Map network is constructed. And analysis about the convergence performance of such a deep Valued Self-Organizing Map network is proved in this paper, which gives an inequality about the designed parameters with the dimension of inputs and the loss of prediction.

Related

• Gradient Normalization & Depth Based Decay For Deep Learning
• Biologically Inspired Feedforward Supervised Learning for Deep Self-Organizing Map Networks
• How neural networks find generalizable solutions: Self-tuned annealing in deep learning
• A Provably Correct Algorithm for Deep Learning that Actually Works
• A Novel Structured Natural Gradient Descent for Deep Learning