Recyclable Gaussian Processes

2020/10/06 by Pablo Moreno-Muñoz, Moreno-Muñoz, Pablo, Antonio Artés-Rodrı́guez +3
Computer Science · Engineering · #Advanced Multi-Objective Optimization Algorithms #Control Systems and Identification #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2010.02554

openalex publication_date 2020/10/06 · openalex created_date 2020/10/15 · openalex updated_date 2026/07/28

Abstract

We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes without revisiting any subset of observations. Our framework allows for regression, classification and heterogeneous tasks, i.e. mix of continuous and discrete variables over the same input domain. We exploit infinite-dimensional integral operators based on the Kullback-Leibler divergence between stochastic processes to re-combine arbitrary amounts of variational sparse approximations with different complexity, likelihood model and location of the pseudo-inputs. Extensive results illustrate the usability of our framework in large-scale distributed experiments, also compared with the exact inference models in the literature.

Citations

• Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions
• Training Products of Experts by Minimizing Contrastive Divergence

Related

• Gaussian Processes for Big Data
• Sparse Orthogonal Variational Inference for Gaussian Processes
• Variational Inference for Gaussian Process Models with Linear Complexity
• Orthogonally Decoupled Variational Gaussian Processes
• Modular Gaussian Processes for Transfer Learning