Quantified advantage of discontinuous weight selection in approximations with deep neural networks

2017/05/03 by Dmitry Yarotsky, Yarotsky, Dmitry · 2 citations
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Image and Signal Denoising Methods #Model Reduction and Neural Networks #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1705.01365

openalex publication_date 2017/05/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider approximations of 1D Lipschitz functions by deep ReLU networks of a fixed width. We prove that without the assumption of continuous weight selection the uniform approximation error is lower than with this assumption at least by a factor logarithmic in the size of the network.

Cited by

• Calibration of a Fluid-Structure Problem with Keras
• Nonlinear Approximation and (Deep) ReLU Networks

Related

• Error bounds for approximations with deep ReLU networks
• Three Quantization Regimes for ReLU Networks
• Optimal approximation of continuous functions by very deep ReLU networks
• Approximation results for Gradient Descent trained Shallow Neural Networks in 1d
• Depth Separations in Neural Networks: Separating the Dimension from the Accuracy