2013/11/28 by Maurizio Filippone, Filippone, Maurizio
Chemistry · Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #Fault Detection and Control Systems #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Methodology (stat.ME) #Spectroscopy and Chemometric Analyses #Statistical Methods and Bayesian Inference
paper · pdf · doi:10.48550/arxiv.1311.7320
openalex publication_date 2013/11/28 · openalex created_date 2022/10/04 · openalex updated_date 2026/07/28
Kernel methods have revolutionized the fields of pattern recognition and\nmachine learning. Their success, however, critically depends on the choice of\nkernel parameters. Using Gaussian process (GP) classification as a working\nexample, this paper focuses on Bayesian inference of covariance (kernel)\nparameters using Markov chain Monte Carlo (MCMC) methods. The motivation is\nthat, compared to standard optimization of kernel parameters, they have been\nsystematically demonstrated to be superior in quantifying uncertainty in\npredictions. Recently, the Pseudo-Marginal MCMC approach has been proposed as a\npractical inference tool for GP models. In particular, it amounts in replacing\nthe analytically intractable marginal likelihood by an unbiased estimate\nobtainable by approximate methods and importance sampling. After discussing the\npotential drawbacks in employing importance sampling, this paper proposes the\napplication of annealed importance sampling. The results empirically\ndemonstrate that compared to importance sampling, annealed importance sampling\ncan reduce the variance of the estimate of the marginal likelihood\nexponentially in the number of data at a computational cost that scales only\npolynomially. The results on real data demonstrate that employing annealed\nimportance sampling in the Pseudo-Marginal MCMC approach represents a step\nforward in the development of fully automated exact inference engines for GP\nmodels.\n