Wasserstein Neural Processes

2019/10/01 by Andrew N. Carr, Carr, Andrew, David Wingate +2
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.1910.00668

openalex publication_date 2019/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Neural Processes (NPs) are a class of models that learn a mapping from a context set of input-output pairs to a distribution over functions. They are traditionally trained using maximum likelihood with a KL divergence regularization term. We show that there are desirable classes of problems where NPs, with this loss, fail to learn any reasonable distribution. We also show that this drawback is solved by using approximations of Wasserstein distance which calculates optimal transport distances even for distributions of disjoint support. We give experimental justification for our method and demonstrate performance. These Wasserstein Neural Processes (WNPs) maintain all of the benefits of traditional NPs while being able to approximate a new class of function mappings.

Citations

• Conditional Neural Processes

Related

• Neural Local Wasserstein Regression
• Approximation Theory, Computing, and Deep Learning on the Wasserstein Space
• Mean-field neural networks: learning mappings on Wasserstein space
• An efficient Wasserstein-distance approach for reconstructing jump-diffusion processes using parameterized neural networks
• A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models