2024/01/12 by Joost A. A. Opschoor, Christoph Schwab, Opschoor, Joost A. A. +3 · 1 citation
Engineering · Physics and Astronomy · #34B08 #34D15 #65L11 #Advanced Numerical Methods in Computational Mathematics #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Numerical Analysis (math.NA) #Numerical methods in engineering
paper · pdf · doi:10.48550/arxiv.2401.06656
openalex publication_date 2024/01/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We prove deep neural network (DNN for short) expressivity rate bounds for solution sets of a model class of singularly perturbed, elliptic two-point boundary value problems, in Sobolev norms, on the bounded interval (-1,1). We assume that the given source term and reaction coefficient are analytic in [-1,1]. We establish expression rate bounds in Sobolev norms in terms of the NN size which are uniform with respect to the singular perturbation parameter for several classes of DNN architectures. In particular, ReLU NNs, spiking NNs, and \tanh- and sigmoid-activated NNs. The latter activations can represent ``exponential boundary layer solution features'' explicitly, in the last hidden layer of the DNN, i.e. in a shallow subnetwork, and afford improved robust expression rate bounds in terms of the NN size. We prove that all DNN architectures allow robust exponential solution expression in so-called `energy' as well as in `balanced' Sobolev norms, for analytic input data.