Preconditioned Additive Gaussian Processes with Fourier Acceleration

2025/04/01 by Wagner, Theresa, Xu, Tianshi, Nestler, Franziska +2
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2504.00480

Abstract

Gaussian processes (GPs) are crucial in machine learning for quantifying uncertainty in predictions. However, their associated covariance matrices, defined by kernel functions, are typically dense and large-scale, posing significant computational challenges. This paper introduces a matrix-free method that utilizes the Non-equispaced Fast Fourier Transform (NFFT) to achieve nearly linear complexity in the multiplication of kernel matrices and their derivatives with vectors for a predetermined accuracy level. To address high-dimensional problems, we propose an additive kernel approach. Each sub-kernel in this approach captures lower-order feature interactions, allowing for the efficient application of the NFFT method and potentially increasing accuracy across various real-world datasets. Additionally, we implement a preconditioning strategy that accelerates hyperparameter tuning, further improving the efficiency and effectiveness of GPs.

Related

• Fast Evaluation of Additive Kernels: Feature Arrangement, Fourier Methods, and Kernel Derivatives
• Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices
• Equispaced Fourier representations for efficient Gaussian process regression from a billion data points
• gp2Scale: A Class of Compactly Supported Non-Stationary Kernels and Distributed Computing for Exact Gaussian Processes on 10 Million Data Points
• Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels