A Stochastic Variance-reduced Accelerated Primal-dual Method for Finite-sum Saddle-point Problems

2020/12/24 by Erfan Yazdandoost Hamedani, Afrooz Jalilzadeh, Hamedani, Erfan Yazdandoost +1 · 1 citation
Computer Science · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Matrix Theory and Algorithms #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2012.13456

openalex publication_date 2020/12/24 · openalex created_date 2023/04/11 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose a variance-reduced primal-dual algorithm with Bregman distance for solving convex-concave saddle-point problems with finite-sum structure and nonbilinear coupling function. This type of problems typically arises in machine learning and game theory. Based on some standard assumptions, the algorithm is proved to converge with oracle complexity of O(\frac√ nε) and O((n)/(√ ε)+\frac1ε^1.5) using constant and non-constant parameters, respectively where n is the number of function components. Compared with existing methods, our framework yields a significant improvement over the number of required primal-dual gradient samples to achieve ε-accuracy of the primal-dual gap. We tested our method for solving a distributionally robust optimization problem to show the effectiveness of the algorithm.

Cited by

• Min-Max Optimization with Dual-Linear Coupling

Related

• Accelerated Stochastic Algorithms for Convex-Concave Saddle-Point Problems
• Accelerated Primal-dual Scheme for a Class of Stochastic Nonconvex-concave Saddle Point Problems
• Stochastic Variance Reduction Methods for Saddle-Point Problems
• A Variance-Reduced Stochastic Accelerated Primal Dual Algorithm
• Variance Reduction via Primal-Dual Accelerated Dual Averaging for\n Nonsmooth Convex Finite-Sums