2024/06/10 by Pierre-François Massiani, Massiani, Pierre-François, Sebastian Trimpe +3 · 1 citation
Computer Science · Mathematics · #FOS: Computer and information sciences #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical methods in inverse problems #Statistical and numerical algorithms
paper · pdf · doi:10.48550/arxiv.2406.06101
openalex publication_date 2024/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The consistency of a learning method is usually established under the assumption that the observations are a realization of an independent and identically distributed (i.i.d.) or mixing process. Yet, kernel methods such as support vector machines (SVMs), Gaussian processes, or conditional kernel mean embeddings (CKMEs) all give excellent performance under sampling schemes that are obviously non-i.i.d., such as when data comes from a dynamical system. We propose the new notion of empirical weak convergence (EWC) as a general assumption explaining such phenomena for kernel methods. It assumes the existence of a random asymptotic data distribution and is a strict weakening of previous assumptions in the field. Our main results then establish consistency of SVMs, kernel mean embeddings, and general Hilbert-space valued empirical expectations with EWC data. Our analysis holds for both finite- and infinite-dimensional outputs, as we extend classical results of statistical learning to the latter case. In particular, it is also applicable to CKMEs. Overall, our results open new classes of processes to statistical learning and can serve as a foundation for a theory of learning beyond i.i.d. and mixing.