Computation complexity of deep ReLU neural networks in high-dimensional approximation

2021/03/01 by Dũng, Dinh, Nguyen, Van Kien, Thao, Mai Xuan
#FOS: Mathematics #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2103.00815

Abstract

The purpose of the present paper is to study the computation complexity of deep ReLU neural networks to approximate functions in Hölder-Nikol'skii spaces of mixed smoothness H_∞^α(\mathbbI^d) on the unit cube \mathbbI^d:=[0,1]^d. In this context, for any function f∈ H_∞^α(\mathbbI^d), we explicitly construct nonadaptive and adaptive deep ReLU neural networks having an output that approximates f with a prescribed accuracy ε, and prove dimension-dependent bounds for the computation complexity of this approximation, characterized by the size and the depth of this deep ReLU neural network, explicitly in d and ε. Our results show the advantage of the adaptive method of approximation by deep ReLU neural networks over nonadaptive one.

Related

• Deep ReLU neural networks in high-dimensional approximation
• Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks
• Optimal approximation of continuous functions by very deep ReLU networks
• Error bounds for approximations with deep ReLU networks
• Optimal approximation of piecewise smooth functions using deep ReLU\n neural networks