Sobolev norm inconsistency of kernel interpolation

2025/04/29 by Yunfei Yang, Yang, Yunfei
Mathematics · #Advanced Harmonic Analysis Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical Approximation and Integration #Spectral Theory in Mathematical Physics

paper · pdf · doi:10.48550/arxiv.2504.20617

openalex publication_date 2025/04/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the consistency of minimum-norm interpolation in reproducing kernel Hilbert spaces corresponding to bounded kernels. Our main result give lower bounds for the generalization error of the kernel interpolation measured in a continuous scale of norms that interpolate between L² and the hypothesis space. These lower bounds imply that kernel interpolation is always inconsistent, when the smoothness index of the norm is larger than a constant that depends only on the embedding index of the hypothesis space and the decay rate of the eigenvalues.

Citations

• Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension
• On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions
• Benign Overfitting in Two-layer Convolutional Neural Networks
• Understanding neural networks with reproducing kernel Banach spaces
• Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation
• Deep learning: a statistical viewpoint
• Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration
• Generalization error of random feature and kernel methods: Hypercontractivity and kernel matrix concentration
• Kernel Interpolation of High Dimensional Scattered Data
• On the Inductive Bias of Neural Tangent Kernels
• Surprises in High-Dimensional Ridgeless Least Squares Interpolation
• Uniform convergence may be unable to explain generalization in deep learning
• On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions
• Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenon
• Reconciling modern machine learning practice and the bias-variance trade-off
• Just Interpolate: Kernel "Ridgeless" Regression Can Generalize
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Understanding deep learning requires rethinking generalization
• Distributed learning with regularized least squares
• Optimal Rates For Regularization Of Statistical Inverse Learning Problems

Related

• Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenon
• Generalization error of minimum weighted norm and kernel interpolation
• Extending error bounds for radial basis function interpolation to measuring the error in higher order Sobolev norms
• Escaping the native space of Sobolev kernels by interpolation
• Kernel interpolation in Sobolev spaces of hybrid regularity