Universal Approximation of Operators with Transformers and Neural Integral Operators

2024/09/01 by Zappala, Emanuele, Bagherian, Maryam · 1 citation
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2409.00841

Abstract

We study the universal approximation properties of transformers and neural integral operators for operators in Banach spaces. In particular, we show that the transformer architecture is a universal approximator of integral operators between Hölder spaces. Moreover, we show that a generalized version of neural integral operators, based on the Gavurin integral, are universal approximators of arbitrary operators between Banach spaces. Lastly, we show that a modified version of transformer, which uses Leray-Schauder mappings, is a universal approximator of operators between arbitrary Banach spaces.

Cited by

• Time Resolution Independent Operator Learning

Related

• A Survey on Universal Approximation Theorems
• Projection Methods for Operator Learning and Universal Approximation
• Nonlocality and Nonlinearity Implies Universality in Operator Learning
• Neural Networks in Fréchet spaces
• Universal approximation theorem for neural networks with inputs from a topological vector space