Error estimate for a universal function approximator of ReLU network with a local connection

2020/09/03 by Jae‐Mo Kang, Jae-Mo Kang, Kang, Jae-Mo +2
Computer Science · Engineering · Mathematics · #Control Systems and Identification #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Microwave Imaging and Scattering Analysis #Neural Networks and Applications #cs.IT #cs.LG #math.IT #stat.ML

paper · pdf · doi:10.48550/arxiv.2009.01461

arxiv created 2020/09/03 · openalex publication_date 2020/09/03 · arxiv updated 2020/09/04 · openalex created_date 2020/09/08 · openalex updated_date 2026/07/28

Abstract

Neural networks have shown high successful performance in a wide range of tasks, but further studies are needed to improve its performance. We analyze the approximation error of the specific neural network architecture with a local connection and higher application than one with the full connection because the local-connected network can be used to explain diverse neural networks such as CNNs. Our error estimate depends on two parameters: one controlling the depth of the hidden layer, and the other, the width of the hidden layers.

Citations

• Identity Matters in Deep Learning
• Approximating Continuous Functions by ReLU Nets of Minimal Width
• Universality of deep convolutional neural networks

Related

• On the growth of the parameters of approximating ReLU neural networks
• Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces
• On the Principles of ReLU Networks with One Hidden Layer
• Error bounds for approximations with deep ReLU networks
• ReLU neural network approximation to piecewise constant functions