Revisiting the Sample Complexity of Sparse Spectrum Approximation of Gaussian Processes

2020/11/17 by Quang Minh Hoang, Trong Nghia Hoang, Hoang, Quang Minh +6 · 2 citations
Biochemistry, Genetics and Molecular Biology · Chemistry · Computer Science · Engineering · Mathematics · #Control Systems and Identification #FOS: Computer and information sciences #Fault Detection and Control Systems #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Spectroscopy Techniques in Biomedical and Chemical Research #Spectroscopy and Chemometric Analyses #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2011.08432

arxiv created 2020/11/17 · openalex publication_date 2020/11/17 · arxiv updated 2020/11/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a new scalable approximation for Gaussian processes with provable guarantees which hold simultaneously over its entire parameter space. Our approximation is obtained from an improved sample complexity analysis for sparse spectrum Gaussian processes (SSGPs). In particular, our analysis shows that under a certain data disentangling condition, an SSGP's prediction and model evidence (for training) can well-approximate those of a full GP with low sample complexity. We also develop a new auto-encoding algorithm that finds a latent space to disentangle latent input coordinates into well-separated clusters, which is amenable to our sample complexity analysis. We validate our proposed method on several benchmarks with promising results supporting our theoretical analysis.

Citations

• Multi-Robot Informative Path Planning for Active Sensing of Environmental Phenomena: A Tale of Two Algorithms
• A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
• GP-Localize: Persistent Mobile Robot Localization using Online Sparse Gaussian Process Observation Model
• Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models
• Decentralized High-Dimensional Bayesian Optimization with Factor Graphs

Cited by

• ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge
• Unpacking the Expressed Consequences of AI Research in Broader Impact Statements

Related

• Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs
• Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels
• gp2Scale: A Class of Compactly Supported Non-Stationary Kernels and Distributed Computing for Exact Gaussian Processes on 10 Million Data Points
• Variational Fourier features for Gaussian processes
• A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression