Infinite Neural Operators: Gaussian processes on functions

2025/10/19 by Daniel Augusto de Souza, Yuchen Zhu, de Souza, Daniel Augusto +9
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.2510.16675

openalex publication_date 2025/10/19 · openalex created_date 2025/10/22 · openalex updated_date 2026/07/28

Abstract

A variety of infinitely wide neural architectures (e.g., dense NNs, CNNs, and transformers) induce Gaussian process (GP) priors over their outputs. These relationships provide both an accurate characterization of the prior predictive distribution and enable the use of GP machinery to improve the uncertainty quantification of deep neural networks. In this work, we extend this connection to neural operators (NOs), a class of models designed to learn mappings between function spaces. Specifically, we show conditions for when arbitrary-depth NOs with Gaussian-distributed convolution kernels converge to function-valued GPs. Based on this result, we show how to compute the covariance functions of these NO-GPs for two NO parametrizations, including the popular Fourier neural operator (FNO). With this, we compute the posteriors of these GPs in regression scenarios, including PDE solution operators. This work is an important step towards uncovering the inductive biases of current FNO architectures and opens a path to incorporate novel inductive biases for use in kernel-based operator learning methods.

Citations

• Linearization Turns Neural Operators into Function-Valued Gaussian Processes
• Kernel Methods are Competitive for Operator Learning
• PDEBENCH: An Extensive Benchmark for Scientific Machine Learning
• Fast Neural Kernel Embeddings for General Activations
• Approximate Bayesian Neural Operators: Uncertainty Quantification for Parametric PDEs
• FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
• Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Gaussian Process Behaviour in Wide Deep Neural Networks
• Deep Neural Networks as Gaussian Processes
• Scaling Multidimensional Inference for Structured Gaussian Processes

Cited by

• Thompson Sampling in Function Spaces via Neural Operators

Related

• Deep Neural Networks as Gaussian Processes
• Linearization Turns Neural Operators into Function-Valued Gaussian Processes
• Operator learning with Gaussian processes
• Neural Processes
• Deep Neural Networks as Point Estimates for Deep Gaussian Processes