2021/02/16 by Jonathan Lorraine, David Acuna, Lorraine, Jonathan +5 · 1 citation
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Computer Science and Game Theory (cs.GT) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.2102.08431
openalex publication_date 2021/02/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We generalize gradient descent with momentum for optimization in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving convergence on bilinear zero-sum games for simultaneous and alternating updates. Our method gives real-valued parameter updates, making it a drop-in replacement for standard optimizers. We empirically demonstrate that complex-valued momentum can improve convergence in realistic adversarial games - like generative adversarial networks - by showing we can find better solutions with an almost identical computational cost. We also show a practical generalization to a complex-valued Adam variant, which we use to train BigGAN to better inception scores on CIFAR-10.