On the inability of Gaussian process regression to optimally learn compositional functions

2022/05/16 by Giordano, Matteo, Ray, Kolyan, Schmidt-Hieber, Johannes · 2 citations
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2205.07764

Abstract

We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

Cited by

• Hierarchical Besov-Laplace priors for spatially inhomogeneous binary classification
• Bayesian inference with Besov-Laplace priors for spatially inhomogeneous binary classification surfaces

Related

• Posterior contraction for deep Gaussian process priors
• Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guarantees
• Posterior Concentration for Gaussian Process Priors under Rescaled and Hierarchical Matérn and Confluent Hypergeometric Covariance Functions
• Posterior contraction in Gaussian process regression using Wasserstein\n approximations
• Posterior contraction rates for constrained deep Gaussian processes in density estimation and classication