Sketching the Heat Kernel: Using Gaussian Processes to Embed Data

2024/03/01 by Anna C. Gilbert, Gilbert, Anna C., Kevin O’Neill +1
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA)

paper · pdf · doi:10.48550/arxiv.2403.07929

openalex publication_date 2024/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces a novel, non-deterministic method for embedding data in low-dimensional Euclidean space based on computing realizations of a Gaussian process depending on the geometry of the data. This type of embedding first appeared in (Adler et al, 2018) as a theoretical model for a generic manifold in high dimensions. In particular, we take the covariance function of the Gaussian process to be the heat kernel, and computing the embedding amounts to sketching a matrix representing the heat kernel. The Karhunen-Loève expansion reveals that the straight-line distances in the embedding approximate the diffusion distance in a probabilistic sense, avoiding the need for sharp cutoffs and maintaining some of the smaller-scale structure. Our method demonstrates further advantage in its robustness to outliers. We justify the approach with both theory and experiments.

Related

• Intrinsic Gaussian Processes on Manifolds and Their Accelerations by Symmetry
• Scalable Bayesian inference for heat kernel Gaussian processes on manifolds
• Intrinsic Gaussian Process on Unknown Manifolds with Probabilistic Metrics
• The GeometricKernels Package: Heat and Matérn Kernels for Geometric Learning on Manifolds, Meshes, and Graphs
• Diffusion Means and Heat Kernel on Manifolds