Distributionally Robust Learning with Weakly Convex Losses: Convergence Rates and Finite-Sample Guarantees

2023/01/16 by Landi Zhu, Zhu, Landi, Mert Gürbüzbalaban +3 · 2 citations
Decision Sciences · Mathematics · #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Risk and Portfolio Optimization #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2301.06619

openalex publication_date 2023/01/16 · openalex created_date 2023/01/20 · openalex updated_date 2026/07/28

Abstract

We consider a distributionally robust stochastic optimization problem and formulate it as a stochastic two-level composition optimization problem with the use of the mean--semideviation risk measure. In this setting, we consider a single time-scale algorithm, involving two versions of the inner function value tracking: linearized tracking of a continuously differentiable loss function, and SPIDER tracking of a weakly convex loss function. We adopt the norm of the gradient of the Moreau envelope as our measure of stationarity and show that the sample complexity of O(ε^-3) is possible in both cases, with only the constant larger in the second case. Finally, we demonstrate the performance of our algorithm with a robust learning example and a weakly convex, non-smooth regression example.

Cited by

• Anchored Langevin Algorithms
• Trust Region Algorithm for Stochastic Minimax Problems with Decision-Dependent Distributions

Related

• A Stochastic Subgradient Method for Distributionally Robust Non-Convex Learning
• Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
• Learning Models with Uniform Performance via Distributionally Robust\n Optimization
• Non-strongly-convex smooth stochastic approximation with convergence rate O(1/n)
• Graphical Convergence of Subgradients in Nonconvex Optimization and\n Learning