Sharp Lower Bounds for Linearized ReLU^k Approximation on the Sphere

2025/10/05 by Mao, Tong, Xu, Jinchao
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2510.04060

Abstract

We prove a saturation theorem for linearized shallow ReLU^k neural networks on the unit sphere \mathbb S^d. For any antipodally quasi-uniform set of centers, if the target function has smoothness r>\tfracd+2k+12, then the best L²(\mathbb S^d) approximation cannot converge faster than order n^{-(d+2k+1)/(2d)}. This lower bound matches existing upper bounds, thereby establishing the exact saturation order \tfracd+2k+12d for such networks. Our results place linearized neural-network approximation firmly within the classical saturation framework and show that, although ReLU^k networks outperform finite elements under equal degrees k, this advantage is intrinsically limited.

Citations

• Integral Representations of Sobolev Spaces via ReLU^k Activation Function and Optimal Error Estimates for Linearized Networks
• Approximation Rates for Shallow ReLU^k Neural Networks on Sobolev Spaces via the Radon Transform
• Highly localized kernels on space of homogeneous type
• Expressivity and Approximation Properties of Deep Neural Networks with ReLU^k Activation
• Tractability of approximation by general shallow networks
• Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks
• Rates of Approximation by ReLU Shallow Neural Networks
• Optimal Convergence Rates for the Orthogonal Greedy Algorithm
• High-Order Approximation Rates for Shallow Neural Networks with Cosine and ReLU^k Activation Functions
• Error bounds for approximations with deep ReLU networks
• Approximation by Combinations of ReLU and Squared ReLU Ridge Functions\n with \ℓ¹ and \ℓ⁰ Controls
• Breaking the Curse of Dimensionality with Convex Neural Networks
• Eignets for function approximation on manifolds
• Sub-exponentially localized kernels and frames induced by orthogonal expansions
• Localized polynomial frames on the interval with Jacobi weights
• Multilayer feedforward networks are universal approximators

Related

• Optimal approximation of continuous functions by very deep ReLU networks
• Approximation Rates for Shallow ReLU^k Neural Networks on Sobolev Spaces via the Radon Transform
• Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks
• Approximation in L^p(μ) with deep ReLU neural networks
• On the Expressive Power of Neural Networks