Deep Stochastic Configuration Networks with Universal Approximation Property

2017/02/18 by Dianhui Wang, Ming Li, Wang, Dianhui +1
Computer Science · #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning and ELM #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1702.05639

openalex publication_date 2017/02/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper develops a randomized approach for incrementally building deep neural networks, where a supervisory mechanism is proposed to constrain the random assignment of the weights and biases, and all the hidden layers have direct links to the output layer. A fundamental result on the universal approximation property is established for such a class of randomized leaner models, namely deep stochastic configuration networks (DeepSCNs). A learning algorithm is presented to implement DeepSCNs with either specific architecture or self-organization. The read-out weights attached with all direct links from each hidden layer to the output layer are evaluated by the least squares method. Given a set of training examples, DeepSCNs can speedily produce a learning representation, that is, a collection of random basis functions with the cascaded inputs together with the read-out weights. An empirical study on a function approximation is carried out to demonstrate some properties of the proposed deep learner model.

Citations

• Stochastic choice of basis functions in adaptive function approximation and the functional-link net
• Functional-link net computing: theory, system architecture, and functionalities
• Deep Learning in Neural Networks: An Overview
• A Fast Learning Algorithm for Deep Belief Nets
• Multilayer feedforward networks are universal approximators
• Provable Bounds for Learning Some Deep Representations
• Representation Learning: A Review and New Perspectives
• The Power of Depth for Feedforward Neural Networks
• A Powerful Generative Model Using Random Weights for the Deep Image Representation

Related

• Deep Stacked Stochastic Configuration Networks for Lifelong Learning of Non-Stationary Data Streams
• Deeper Insights into Learning Performance of Stochastic Configuration Networks
• Deep Recurrent Stochastic Configuration Networks for Modelling Nonlinear Dynamic Systems
• Stochastic Feedforward Neural Networks: Universal Approximation
• Intriguing Properties of Randomly Weighted Networks: Generalizing While\n Learning Next to Nothing