Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty

2023/10/31 by Lukas Tatzel, Jonathan Wenger, Tatzel, Lukas +5
Decision Sciences · Engineering · Computer Science · #Simulation Techniques and Applications #Fault Detection and Control Systems #Target Tracking and Data Fusion in Sensor Networks

paper · pdf · doi:10.48550/arxiv.2310.20285

Abstract

Non-conjugate Gaussian processes (NCGPs) define a flexible probabilistic framework to model categorical, ordinal and continuous data, and are widely used in practice. However, exact inference in NCGPs is prohibitively expensive for large datasets, thus requiring approximations in practice. The approximation error adversely impacts the reliability of the model and is not accounted for in the uncertainty of the prediction. We introduce a family of iterative methods that explicitly model this error. They are uniquely suited to parallel modern computing hardware, efficiently recycle computations, and compress information to reduce both the time and memory requirements for NCGPs. As we demonstrate on large-scale classification problems, our method significantly accelerates posterior inference compared to competitive baselines by trading off reduced computation for increased uncertainty.

Related

• Posterior and Computational Uncertainty in Gaussian Processes
• Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference
• Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
• Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models
• Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference