Kernel methods in machine learning

2007/01/31 by Thomas Hofmann, Bernhard Schölkopf, Alexander J. Smola
Computer Science · Mathematics · #Gaussian Processes and Bayesian Inference #Statistical Methods and Inference #Stochastic Gradient Optimization Techniques #math.PR #math.ST #msc:30C40 #msc:68T05 #stat.TH

paper · pdf · doi:10.1214/009053607000000677

published as Annals of Statistics 2008, Vol. 36, No. 3, 1171-1220 · Published in at http://dx.doi.org/10.1214/009053607000000677 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2008/05/26 · arxiv created 2008/07/01 · arxiv updated 2009/12/01 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/01

Abstract

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

Citations

• Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
• Atomic Decomposition by Basis Pursuit
• The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
• Estimating the Support of a High-Dimensional Distribution
• Training Invariant Support Vector Machines
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities