Analysis of Deep Neural Networks with Quasi-optimal polynomial approximation rates

2019/12/04 by Joseph Daws, Daws, Joseph, Clayton Webster +1 · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #65D15 #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Numerical Analysis (math.NA) #Numerical Methods and Algorithms #cs.LG #cs.NA #math.NA #msc:65D15

paper · pdf · doi:10.48550/arxiv.1912.02302

13 pages submitted to MSML 2020

arxiv created 2019/12/04 · openalex publication_date 2019/12/04 · arxiv updated 2019/12/09 · openalex created_date 2019/12/13 · openalex updated_date 2026/07/28

Abstract

We show the existence of a deep neural network capable of approximating a wide class of high-dimensional approximations. The construction of the proposed neural network is based on a quasi-optimal polynomial approximation. We show that this network achieves an error rate that is sub-exponential in the number of polynomial functions, M, used in the polynomial approximation. The complexity of the network which achieves this sub-exponential rate is shown to be algebraic in M.

Citations

• Long Short-Term Memory
• A Fast Learning Algorithm for Deep Belief Nets
• ImageNet classification with deep convolutional neural networks

Cited by

• SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures

Related

• SUPN: Shallow Universal Polynomial Networks
• Deep Neural Network Approximation Theory
• Nonlinear Approximation and (Deep) ReLU Networks
• Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks
• Expressivity of Shallow and Deep Neural Networks for Polynomial Approximation