Dual Parameterization of Sparse Variational Gaussian Processes

2021/11/05 by Vincent Adam, Paul E. Chang, Adam, Vincent +5 · 4 citations
Biochemistry, Genetics and Molecular Biology · Chemistry · Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Spectroscopy Techniques in Biomedical and Chemical Research #Spectroscopy and Chemometric Analyses

paper · pdf · doi:10.48550/arxiv.2111.03412

openalex publication_date 2021/11/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.

Cited by

• Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
• SING: SDE Inference via Natural Gradients
• Knowledge Adaptation as Posterior Correction
• Federated ADMM from Bayesian Duality

Related

• Sparse Orthogonal Variational Inference for Gaussian Processes
• Tighter sparse variational Gaussian processes
• New Bounds for Sparse Variational Gaussian Processes
• Sparse Gaussian Processes Revisited: Bayesian Approaches to\n Inducing-Variable Approximations
• MCMC for Variationally Sparse Gaussian Processes