2021/09/01 by Sean Hon, Haizhao Yang, Hon, Sean +1 · 3 citations
Computer Science · Engineering · Mathematics · Physics and Astronomy · #Advanced Numerical Methods in Computational Mathematics #FOS: Mathematics #Model Reduction and Neural Networks #Numerical Analysis (math.NA) #Numerical methods in engineering #cs.NA #math.NA
paper · pdf · doi:10.48550/arxiv.2109.00161
openalex publication_date 2021/09/01 · arxiv created 2022/07/22 · arxiv updated 2022/07/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We establish in this work approximation results of deep neural networks for smooth functions measured in Sobolev norms, motivated by recent development of numerical solvers for partial differential equations using deep neural networks. Our approximation results are nonasymptotic in the sense that the error bounds are explicitly characterized in terms of both the width and depth of the networks simultaneously with all involved constants explicitly determined. Namely, for f∈ Cs([0,1]d), we show that deep ReLU networks of width O(NlogN) and of depth O(LlogL) can achieve a nonasymptotic approximation rate of O(N-2(s-1)/dL-2(s-1)/d) with respect to the W1,p([0,1]d) norm for p∈[1,∞). If either the ReLU function or its square is applied as activation functions to construct deep neural networks of width O(NlogN) and of depth O(LlogL) to approximate f∈ Cs([0,1]d), the approximation rate is O(N-2(s-n)/dL-2(s-n)/d) with respect to the Wn,p([0,1]d) norm for p∈[1,∞).