A representer theorem for deep kernel learning

2017/09/29 by Bastian Bohn, Bohn, Bastian, Michael Griebel +3 · 4 citations
Computer Science · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Mathematics #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Mathematical Approximation and Integration #Model Reduction and Neural Networks #Numerical Analysis (math.NA)

paper · pdf · doi:10.48550/arxiv.1709.10441

openalex publication_date 2017/09/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we provide a finite-sample and an infinite-sample representer theorem for the concatenation of (linear combinations of) kernel functions of reproducing kernel Hilbert spaces. These results serve as mathematical foundation for the analysis of machine learning algorithms based on compositions of functions. As a direct consequence in the finite-sample case, the corresponding infinite-dimensional minimization problems can be recast into (nonlinear) finite-dimensional minimization problems, which can be tackled with nonlinear optimization algorithms. Moreover, we show how concatenated machine learning problems can be reformulated as neural networks and how our representer theorem applies to a broad class of state-of-the-art deep learning methods.

Cited by

• Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning
• Feature Understanding and Sparsity Enhancement via 2-Layered kernel machines (2L-FUSE)
• Solving Approximation Tasks with Greedy Deep Kernel Methods
• LMKL-Net: A Fast Localized Multiple Kernel Learning Solver via Deep Neural Networks

Related

• Neural reproducing kernel Banach spaces and representer theorems for deep networks
• A Generalized Representer Theorem for Hilbert Space - Valued Functions
• Analysis of Structured Deep Kernel Networks
• Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime
• Hypothesis Spaces for Deep Learning