2016/07/08 by Gilles Blanchard, Nicole Krämer, Blanchard, Gilles +1 · 2 citations
Engineering · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Numerical methods in inverse problems #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1607.02387
openalex publication_date 2016/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We prove statistical rates of convergence for kernel-based least squares\nregression from i.i.d. data using a conjugate gradient algorithm, where\nregularization against overfitting is obtained by early stopping. This method\nis related to Kernel Partial Least Squares, a regression method that combines\nsupervised dimensionality reduction with least squares projection. Following\nthe setting introduced in earlier related literature, we study so-called "fast\nconvergence rates" depending on the regularity of the target regression\nfunction (measured by a source condition in terms of the kernel integral\noperator) and on the effective dimensionality of the data mapped into the\nkernel space. We obtain upper bounds, essentially matching known minimax lower\nbounds, for the \L2 (prediction) norm as well as for the stronger\nHilbert norm, if the true regression function belongs to the reproducing kernel\nHilbert space. If the latter assumption is not fulfilled, we obtain similar\nconvergence rates for appropriate norms, provided additional unlabeled data are\navailable.\n