Optimality conditions for nonsmooth nonconvex-nonconcave min-max problems and generative adversarial networks

2022/03/21 by Jie Jiang, Xiaojun Chen, Jiang, Jie +1 · 2 citations
Computer Science · Mathematics · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Generative Adversarial Networks and Image Synthesis #Model Reduction and Neural Networks #math.OC #msc:65K15 #msc:90C15 #msc:90C33 #msc:90C47

paper · pdf · doi:10.48550/arxiv.2203.10914

arxiv created 2022/03/21 · arxiv updated 2022/03/22

Abstract

This paper considers a class of nonsmooth nonconvex-nonconcave min-max problems in machine learning and games. We first provide sufficient conditions for the existence of global minimax points and local minimax points. Next, we establish the first-order and second-order optimality conditions for local minimax points by using directional derivatives. These conditions reduce to smooth min-max problems with Fréchet derivatives. We apply our theoretical results to generative adversarial networks (GANs) in which two neural networks contest with each other in a game. Examples are used to illustrate applications of the new theory for training GANs.

Cited by

• Solutions of Two-stage Stochastic Minimax Problems
• Nonconvex Nonsmooth Multicomposite Optimization and Its Applications to Recurrent Neural Networks

Related

• What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization?
• Minimax Optimization with Smooth Algorithmic Adversaries
• A Limited-Capacity Minimax Theorem for Non-Convex Games or: How I Learned to Stop Worrying about Mixed-Nash and Love Neural Nets
• Optimality Conditions for Constrained Minimax Optimization
• Generative Minimization Networks: Training GANs Without Competition