On the Universal Approximation Property of Deep Fully Convolutional Neural Networks

2022/11/25 by Ting Lin, Zuowei Shen, Lin, Ting +3 · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Tensor decomposition and applications

paper · pdf · doi:10.48550/arxiv.2211.14047

openalex publication_date 2022/11/25 · openalex created_date 2022/11/30 · openalex updated_date 2026/07/28

Abstract

We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual fully convolutional networks and their continuous-layer counterpart can achieve universal approximation of these symmetric functions at constant channel width. Moreover, we show that the same can be achieved by non-residual variants with at least 2 channels in each layer and convolutional kernel size of at least 2. In addition, we show that these requirements are necessary, in the sense that networks with fewer channels or smaller kernels fail to be universal approximators.

Cited by

• A unified framework for establishing the universal approximation of transformer-type architectures

Related

• Universality of Deep Convolutional Neural Networks
• Universal Approximation Property of Fully Convolutional Neural Networks with Zero Padding
• Deep vs. shallow networks : An approximation theory perspective
• The universal approximation power of finite-width deep ReLU networks
• Deep Neural Network Approximation of Invariant Functions through Dynamical Systems