On the Optimality of Misspecified Kernel Ridge Regression

2023/05/12 by Zhang, Haobo, Li, Yicheng, Lu, Weihao +1 · 1 citation
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2305.07241

Abstract

In the misspecified kernel ridge regression problem, researchers usually assume the underground true function f_ρ^* ∈ [H]^s, a less-smooth interpolation space of a reproducing kernel Hilbert space (RKHS) H for some s∈ (0,1). The existing minimax optimal results require ‖f_ρ^*‖_L^∞<∞ which implicitly requires s > α₀ where α₀∈ (0,1) is the embedding index, a constant depending on H. Whether the KRR is optimal for all s∈ (0,1) is an outstanding problem lasting for years. In this paper, we show that KRR is minimax optimal for any s∈ (0,1) when the H is a Sobolev RKHS.

Cited by

• Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimation

Related

• When Kernel Ridge Regression Meets the Hölder-Zygmund Class: Minimax Optimality and Failure of Properness
• Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensions
• On the kernel learning problem
• Optimal Rates and Saturation for Noiseless Kernel Ridge Regression
• Fixing an error in Caponnetto and de Vito (2007)