Limitations on approximation by deep and shallow neural networks

2022/11/30 by Petrova, Guergana, Wojtaszczyk, Przemysław · 1 citation
#FOS: Computer and information sciences #FOS: Mathematics #Functional Analysis (math.FA) #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2212.02223

Abstract

We prove Carl's type inequalities for the error of approximation of compact sets K by deep and shallow neural networks. This in turn gives lower bounds on how well we can approximate the functions in K when requiring the approximants to come from outputs of such networks. Our results are obtained as a byproduct of the study of the recently introduced Lipschitz widths.

Cited by

• Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters

Related

• Neural networks: deep, shallow, or in between?
• An in-depth look at approximation via deep and narrow neural networks
• A Numerical Investigation of the Minimum Width of a Neural Network
• A General Constructive Upper Bound on Shallow Neural Nets Complexity
• Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality