Gradient Flow Sampler-based Distributionally Robust Optimization

2025/10/29 by Z. K. Xu, Zusen Xu, Xu, Zusen +3 · 1 voice
Computer Science · Decision Sciences · Mathematics · #Advanced Optimization Algorithms Research #Analysis of PDEs (math.AP) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Stochastic Gradient Optimization Techniques #math.AP #math.OC #stat.ML

paper · pdf · doi:10.48550/arxiv.2510.25956

openalex publication_date 2025/10/29 · arxiv published 2025/10/29 · openalex created_date 2025/11/01 · arxiv updated 2026/05/25 · openalex updated_date 2026/07/28

Abstract

We propose a mathematically principled PDE gradient flow framework for distributionally robust optimization (DRO). Exploiting the recent advances in the intersection of Markov Chain Monte Carlo sampling and gradient flow theory, we show that our theoretical framework can be implemented as practical algorithms for sampling from worst-case distributions and, consequently, DRO. While numerous previous works have proposed various reformulation techniques and iterative algorithms, we contribute a sound gradient flow view of the distributional optimization that can be used to construct new algorithms. As an example of applications, we solve a class of Wasserstein and Sinkhorn DRO problems using the recently-discovered Wasserstein Fisher-Rao and Stein variational gradient flows. Notably, we also show some simple reductions of our framework recover exactly previously proposed popular DRO methods, and provide new insights into their theoretical limit and optimization dynamics. Numerical studies based on stochastic gradient descent provide empirical backing for our theoretical findings.

Citations

• Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimization
• Some notes on the Hellinger distance and various Fisher-Rao distances
• Mixing Time of the Proximal Sampler in Relative Fisher Information via Strong Data Processing Inequality
• The Performance Of The Unadjusted Langevin Algorithm Without Smoothness Assumptions
• Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionals
• Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces
• Statistical optimal transport
• Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalities
• Flow-based Distributionally Robust Optimization
• Sampling via Gradient Flows in the Space of Probability Measures
• Strategic Distribution Shift of Interacting Agents via Coupled Gradient Flows
• An introduction to the analysis of gradients systems
• On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it
• A Finite-Particle Convergence Rate for Stein Variational Gradient Descent
• An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games
• Entropy-regularized Wasserstein distributionally robust shape and topology optimization
• Improved analysis for a proximal algorithm for sampling
• Sinkhorn Distributionally Robust Optimization
• A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1
• The query complexity of sampling from strongly log-concave distributions in one dimension
• Fast Distributionally Robust Learning with Variance Reduced Min-Max Optimization
• Large-Scale Methods for Distributionally Robust Optimization
• Structured Logconcave Sampling with a Restricted Gaussian Oracle
• Adversarial Latent Autoencoders
• Accelerating Langevin Sampling with Birth-death
• Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Continuum Limit of Posteriors in Graph Bayesian Inverse Problems
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
• Distributionally Robust Stochastic Optimization with Wasserstein Distance
• Distributionally Robust Stochastic Optimization with Wasserstein Distance
• Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations
• Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
• Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming
• Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality

Discussions

• www.arxiv.org/abs/2510.25956 'Gradient Flow Sampler-based Distributionally Robust Optimization' - Zusen Xu, Jia-Jie Zhu Can ideas from the world of Wasserstein gradient flows be used to solve problems [bsky, 1 points, 0 comments]

Related

• Flow-based Distributionally Robust Optimization
• Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
• From Optimization to Sampling Through Gradient Flows
• Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimization
• Sampling via Gradient Flows in the Space of Probability Measures