Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space

2020/09/23 by Kazuma Tsuji, Taiji Suzuki, Tsuji, Kazuma +1 · 1 citation
Mathematics · Medicine · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical Analysis and Transform Methods #Mathematical Approximation and Integration #Medical Imaging Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2009.11285

openalex publication_date 2020/09/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep learning has achieved notable success in various fields, including image and speech recognition. One of the factors in the successful performance of deep learning is its high feature extraction ability. In this study, we focus on the adaptivity of deep learning; consequently, we treat the variable exponent Besov space, which has a different smoothness depending on the input location x. In other words, the difficulty of the estimation is not uniform within the domain. We analyze the general approximation error of the variable exponent Besov space and the approximation and estimation errors of deep learning. We note that the improvement based on adaptivity is remarkable when the region upon which the target function has less smoothness is small and the dimension is large. Moreover, the superiority to linear estimators is shown with respect to the convergence rate of the estimation error.

Citations

• Decision theoretic generalizations of the PAC model for neural net and other learning applications
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• ImageNet classification with deep convolutional neural networks
• Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space

Cited by

• Sharp uniform approximation for spectral Barron functions by deep neural networks

Related

• Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality
• Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
• Function space analysis of deep learning representation layers
• A new approach to generalisation error of machine learning algorithms: Estimates and convergence
• Fast learning rate of deep learning via a kernel perspective