Data Complexity Estimates for Operator Learning

2024/05/25 by Nikola Kovachki, Samuel Lanthaler, Kovachki, Nikola B. +3 · 6 citations
Computer Science · Engineering · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #FOS: Mathematics #Fault Detection and Control Systems #Machine Learning (cs.LG) #Machine Learning and Data Classification #Numerical Analysis (math.NA)

paper · pdf · doi:10.48550/arxiv.2405.15992

openalex publication_date 2024/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Operator learning has emerged as a new paradigm for the data-driven approximation of nonlinear operators. Despite its empirical success, the theoretical underpinnings governing the conditions for efficient operator learning remain incomplete. The present work develops theory to study the data complexity of operator learning, complementing existing research on the parametric complexity. We investigate the fundamental question: How many input/output samples are needed in operator learning to achieve a desired accuracy ε? This question is addressed from the point of view of n-widths, and this work makes two key contributions. The first contribution is to derive lower bounds on n-widths for general classes of Lipschitz and Fréchet differentiable operators. These bounds rigorously demonstrate a ``curse of data-complexity'', revealing that learning on such general classes requires a sample size exponential in the inverse of the desired accuracy ε. The second contribution of this work is to show that ``parametric efficiency'' implies ``data efficiency''; using the Fourier neural operator (FNO) as a case study, we show rigorously that on a narrower class of operators, efficiently approximated by FNO in terms of the number of tunable parameters, efficient operator learning is attainable in data complexity as well. Specifically, we show that if only an algebraically increasing number of tunable parameters is needed to reach a desired approximation accuracy, then an algebraically bounded number of data samples is also sufficient to achieve the same accuracy.

Cited by

• Towards Sharp Minimax Risk Bounds for Operator Learning
• PIP² Net: Physics-informed Partition Penalty Deep Operator Network
• Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictors
• Fourier Neural Operators for Time-Periodic Quantum Systems: Learning Floquet Hamiltonians, Observable Dynamics, and Operator Growth
• An Evolutionary Multi-objective Optimization for Replica-Exchange-based Physics-informed Operator Learning Network
• Delay-adaptive Control of Nonlinear Systems with Approximate Neural Operator Predictors
• Diffeomorphic Neural Operator Learning

Related

• Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective
• The Parametric Complexity of Operator Learning
• Theory-to-Practice Gap for Neural Networks and Neural Operators
• Bounding the Rademacher Complexity of Fourier neural operators
• Mixture of Experts Softens the Curse of Dimensionality in Operator Learning