A piece-wise constant approximation for non-conjugate Gaussian Process models

2022/04/22 by Seitz, Sarem
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2204.10575

Abstract

Gaussian Processes (GPs) are a versatile and popular method in Bayesian Machine Learning. A common modification are Sparse Variational Gaussian Processes (SVGPs) which are well suited to deal with large datasets. While GPs allow to elegantly deal with Gaussian-distributed target variables in closed form, their applicability can be extended to non-Gaussian data as well. These extensions are usually impossible to treat in closed form and hence require approximate solutions. This paper proposes to approximate the inverse-link function, which is necessary when working with non-Gaussian likelihoods, by a piece-wise constant function. It will be shown that this yields a closed form solution for the corresponding SVGP lower bound. In addition, it is demonstrated how the piece-wise constant function itself can be optimized, resulting in an inverse-link function that can be learnt from the data at hand.

Related

• MCMC for Variationally Sparse Gaussian Processes
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• Sparse Orthogonal Variational Inference for Gaussian Processes
• Non-Parametric Variational Inference with Graph Convolutional Networks for Gaussian Processes
• Sparse Gaussian Processes Revisited: Bayesian Approaches to\n Inducing-Variable Approximations