MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets

2018/11/30 by Corentin Hardy, Erwan Le Merrer, Bruno Sericola +1 · 204 citations
Computer Science · Mathematics · #Artificial intelligence #Artificial neural network #Computer science #Deep learning #Discriminator #Domain (mathematical analysis) #Face recognition and analysis #Generative adversarial network #Generative grammar #MNIST database #Machine learning #Node (physics) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #Train #cs.LG #stat.ML

paper · pdf · doi:10.1109/ipdps.2019.00095

published as IEEE International Parallel and Distributed Processing Symposium (IPDPS) 2019 · To be published in IPDPS 2019: the 33rd IEEE International Parallel & Distributed Processing Symposium

openalex created_date 2018/11/16 · arxiv created 2019/02/06 · openalex publication_date 2019/05/01 · arxiv updated 2021/04/14 · openalex updated_date 2026/08/05

Abstract

A recent technical breakthrough in the domain of machine learning is the discovery and the multiple applications of Generative Adversarial Networks (GANs). Those generative models are computationally demanding, as a GAN is composed of two deep neural networks, and because it trains on large datasets. A GAN is generally trained on a single server. In this paper, we address the problem of distributing GANs so that they are able to train over datasets that are spread on multiple workers. MD-GAN is exposed as the first solution for this problem: we propose a novel learning procedure for GANs so that they fit this distributed setup. We then compare the performance of MD-GAN to an adapted version of federated learning to GANs, using the MNIST, CIFAR10 and CelebA datasets. MD-GAN exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better or identical performances with the adaptation of federated learning. We finally discuss the practical implications of distributing GANs.

Citations

• Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
• Adam: A Method for Stochastic Optimization
• Generative Adversarial Networks
• Dynamo
• Towards Geo-Distributed Machine Learning
• Generative Adversarial Text to Image Synthesis
• Federated Learning: Strategies for Improving Communication Efficiency
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Invertible Conditional GANs for image editing
• Generative Multi-Adversarial Networks
• Gossip training for deep learning
• Ensembles of Generative Adversarial Networks
• Generative Adversarial Parallelization
• AdaGAN: Boosting Generative Models
• Towards Principled Methods for Training Generative Adversarial Networks
• Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently
• Multi-Generator Generative Adversarial Nets
• Generating Videos with Scene Dynamics
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• TensorFlow: A system for large-scale machine learning
• Learning Multiple Layers of Features from Tiny Images
• Generative Adversarial Networks

Cited by

• Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation