Extending Universal Approximation Guarantees: A Theoretical Justification for the Continuity of Real-World Learning Tasks

2022/12/06 by Durvasula, Naveen
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probability (math.PR)

paper · doi:10.48550/arxiv.2212.07934

Abstract

Universal Approximation Theorems establish the density of various classes of neural network function approximators in C(K, ℝ^m), where K ⊂ ℝⁿ is compact. In this paper, we aim to extend these guarantees by establishing conditions on learning tasks that guarantee their continuity. We consider learning tasks given by conditional expectations x ↦ E[Y | X = x], where the learning target Y = f ∘ L is a potentially pathological transformation of some underlying data-generating process L. Under a factorization L = T ∘ W for the data-generating process where T is thought of as a deterministic map acting on some random input W, we establish conditions (that might be easily verified using knowledge of T alone) that guarantee the continuity of practically any derived learning task x ↦ E[f ∘ L | X = x]. We motivate the realism of our conditions using the example of randomized stable matching, thus providing a theoretical justification for the continuity of real-world learning tasks.

Related

• Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data
• On Universal Approximation by Neural Networks with Uniform Guarantees on Approximation of Infinite Dimensional Maps
• An elementary proof of a universal approximation theorem
• Approximation capability of neural networks on spaces of probability measures and tree-structured domains
• Universal Approximation Theorem for Neural Networks