Lipschitz widths

2021/11/02 by Petrova, Guergana, Wojtaszczyk, Przemyslaw · 2 citations
#FOS: Mathematics #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2111.01341

Abstract

This paper introduces a measure, called Lipschitz widths, of the optimal performance possible of certain nonlinear methods of approximation. It discusses their relation to entropy numbers and other well known widths such as the Kolmogorov and the stable manifold widths. It also shows that the Lipschitz widths provide a theoretical benchmark for the approximation quality achieved via deep neural networks.

Cited by

• Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data
• Sharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters

Related

• Optimal Stable Nonlinear Approximation
• On the entropy numbers and the Kolmogorov widths
• Neural networks: deep, shallow, or in between?
• A Numerical Investigation of the Minimum Width of a Neural Network
• Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels