Revisiting Active Sets for Gaussian Process Decoders

2022/09/10 by Pablo Moreno-Muñoz, Moreno-Muñoz, Pablo, Cilie W. Feldager +3 · 2 citations
Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2209.04636

openalex publication_date 2022/09/10 · openalex created_date 2022/09/14 · openalex updated_date 2026/07/28

Abstract

Decoders built on Gaussian processes (GPs) are enticing due to the marginalisation over the non-linear function space. Such models (also known as GP-LVMs) are often expensive and notoriously difficult to train in practice, but can be scaled using variational inference and inducing points. In this paper, we revisit active set approximations. We develop a new stochastic estimate of the log-marginal likelihood based on recently discovered links to cross-validation, and propose a computationally efficient approximation thereof. We demonstrate that the resulting stochastic active sets (SAS) approximation significantly improves the robustness of GP decoder training while reducing computational cost. The SAS-GP obtains more structure in the latent space, scales to many datapoints and learns better representations than variational autoencoders, which is rarely the case for GP decoders.

Cited by

• Atlas Gaussian processes on restricted domains and point clouds
• Intrinsic-Hybrid Latent Diffusion Models for Generative Modeling on Unknown Manifolds

Related

• Predictive Active Set Selection Methods for Gaussian Processes
• Active Learning of Linear Embeddings for Gaussian Processes
• Factorized Gaussian Process Variational Autoencoders
• Scalable Gaussian Process Variational Autoencoders
• Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling