Adaptive sparse variational approximations for Gaussian process regression

2025/04/04 by Dennis Nieman, Nieman, Dennis, Botond Szabó +1 · 2 citations
Computer Science · Mathematics · #Advanced Multi-Objective Optimization Algorithms #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2504.03321

openalex publication_date 2025/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Accurate tuning of hyperparameters is crucial to ensure that models can generalise effectively across different settings. In this paper, we present theoretical guarantees for hyperparameter selection using variational Bayes in the nonparametric regression model. We construct a variational approximation to a hierarchical Bayes procedure, and derive upper bounds for the contraction rate of the variational posterior in an abstract setting. The theory is applied to various Gaussian process priors and variational classes, resulting in minimax optimal rates. Our theoretical results are accompanied with numerical analysis both on synthetic and real world data sets.

Cited by

• Consistency of variational inference for Besov priors in non-linear inverse problems
• Adaptive Resolution for Finite-Rank Gaussian Processes

Related

• Contraction rates for sparse variational approximations in Gaussian\n process regression
• New Bounds for Sparse Variational Gaussian Processes
• Tighter sparse variational Gaussian processes
• Rates of Convergence for Sparse Variational Gaussian Process Regression
• Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression