Scalable Cross Validation Losses for Gaussian Process Models

2021/05/24 by Martin Jankowiak, Geoff Pleiss, Jankowiak, Martin +1 · 2 citations
Computer Science · #Advanced Multi-Objective Optimization Algorithms #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2105.11535

openalex publication_date 2021/05/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation. To accommodate binary and multi-class classification we leverage Pòlya-Gamma auxiliary variables and variational inference. In an extensive empirical comparison with a number of alternative methods for scalable GP regression and classification, we find that our method offers fast training and excellent predictive performance. We argue that the good predictive performance can be traced to the non-parametric nature of the resulting predictive distributions as well as to the cross-validation loss, which provides robustness against model mis-specification.

Citations

• Adam: A Method for Stochastic Optimization
• Scalable Variational Gaussian Process Classification
• Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
• Multidimensional binary search trees used for associative searching

Cited by

• Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits
• Fast and Scalable Spike and Slab Variable Selection in High-Dimensional\n Gaussian Processes

Related

• Modulating Scalable Gaussian Processes for Expressive Statistical Learning
• Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods
• Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation
• Comparing Scale Parameter Estimators for Gaussian Process Interpolation with the Brownian Motion Prior: Leave-One-Out Cross Validation and Maximum Likelihood
• Quadruply Stochastic Gaussian Processes