Node-By-Node Greedy Deep Learning for Interpretable Features

2016/02/19 by Ke Wu, Wu, Ke, Malik Magdon‐Ismail +2 · 1 citation
Computer Science · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Speech Recognition and Synthesis #cs.LG

paper · pdf · doi:10.48550/arxiv.1602.06183

arxiv created 2016/02/19 · openalex publication_date 2016/02/19 · arxiv updated 2016/02/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multilayer networks have seen a resurgence under the umbrella of deep learning. Current deep learning algorithms train the layers of the network sequentially, improving algorithmic performance as well as providing some regularization. We present a new training algorithm for deep networks which trains each node in the network sequentially. Our algorithm is orders of magnitude faster, creates more interpretable internal representations at the node level, while not sacrificing on the ultimate out-of-sample performance.

Citations

• The perceptron: A probabilistic model for information storage and organization in the brain.
• A Fast Learning Algorithm for Deep Belief Nets
• ImageNet classification with deep convolutional neural networks

Cited by

• Network Signatures from Image Representation of Adjacency Matrices: Deep/Transfer Learning for Subgraph Classification

Related

• MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
• Deep Networks from the Principle of Rate Reduction
• LayerDropBack: A Universally Applicable Approach for Accelerating Training of Deep Networks
• Provable Bounds for Learning Some Deep Representations
• Faster learning of deep stacked autoencoders on multi-core systems using synchronized layer-wise pre-training