NIPS 2016 Tutorial: Generative Adversarial Networks

2016/12/31 by Ian Goodfellow, Goodfellow, Ian · 4 voices · 189 citations
Computer Science · #cs.LG

paper · pdf · doi:10.48550/arxiv.1701.00160

v2-v4 are all typo fixes. No substantive changes relative to v1

arxiv created 2017/04/03 · arxiv updated 2017/04/05

Abstract

This report summarizes the tutorial presented by the author at NIPS 2016 on generative adversarial networks (GANs). The tutorial describes: (1) Why generative modeling is a topic worth studying, (2) how generative models work, and how GANs compare to other generative models, (3) the details of how GANs work, (4) research frontiers in GANs, and (5) state-of-the-art image models that combine GANs with other methods. Finally, the tutorial contains three exercises for readers to complete, and the solutions to these exercises.

Citations

• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Cited by

• Exploring Bias in GAN-based Data Augmentation for Small Samples
• Typeface Completion with Generative Adversarial Networks
• A Wasserstein Minimax Framework for Mixed Linear Regression
• Training Generative Adversarial Networks via Primal-Dual Subgradient Methods: A Lagrangian Perspective on GAN
• Mosaic: A Sample-Based Database System for Open World Query Processing
• HumanGAN: generative adversarial network with human-based discriminator and its evaluation in speech perception modeling
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Synthetic Data -- Anonymisation Groundhog Day
• Improved Training of Generative Adversarial Networks Using Representative Features
• Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training
• An Uncertainty-aware Hierarchical Probabilistic Network for Early Prediction, Quantification and Segmentation of Pulmonary Tumour Growth
• Approximate Query Processing using Deep Generative Models
• Generative Model Predictive Control in Manufacturing Processes: A Review
• Adversarial Counterfactual Learning and Evaluation for Recommender System
• A Probabilistic U-Net for Segmentation of Ambiguous Images
• FitVid: Overfitting in Pixel-Level Video Prediction
• Statistical guarantees for generative models without domination
• Advancements in synthetic data extraction for industrial injection molding
• Toward Multimodal Image-to-Image Translation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Is It a Plausible Colour? UCapsNet for Image Colourisation
• An Introduction to Image Synthesis with Generative Adversarial Nets
• A Novel Framework for Selection of GANs for an Application
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks
• On the Conceptualization of Synthetic Data: Why Not All Synthetic Data are Useful and Some are
• Multimodal Sensor Fusion In Single Thermal image Super-Resolution
• Mode Collapse and Regularity of Optimal Transportation Maps
• Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel
• MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation
• An Improved Self-supervised GAN via Adversarial Training
• On the Origin of Deep Learning
• Privacy preserving Generative Adversarial Networks to model Electronic Health Records
• DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion
• Exploring Generative Adversarial Networks for Image-to-Image Translation\n in STEM Simulation
• Understanding the Effectiveness of Lipschitz-Continuity in Generative Adversarial Nets
• AXNet: ApproXimate computing using an end-to-end trainable neural network
• A Variational Inequality Perspective on Generative Adversarial Networks
• Exponential Convergence of Gradient Methods in Concave Network Zero-sum Games
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• PCP-GAN: Property-Constrained Pore-scale image reconstruction via conditional Generative Adversarial Networks
• CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
• Investigating Under and Overfitting in Wasserstein Generative Adversarial Networks
• One Pass Is Not Enough: Recursive Latent Refinement for Generative Models
• Generative Latent Flow
• Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs
• Some Theoretical Properties of GANs
• mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations
• Facing & mitigating common challenges when working with real-world data: The Data Learning Paradigm
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Understanding and Stabilizing GANs' Training Dynamics with Control Theory
• Diagnosing the Impact of AI on Radiology in China
• Semi-Implicit Generative Model
• On Convergence and Stability of GANs
• Privacy-preserving Federated Bayesian Learning of a Generative Model for Imbalanced Classification of Clinical Data
• Jointly Measuring Diversity and Quality in Text Generation Models
• Conditional Generative Neural System for Probabilistic Trajectory Prediction
• Face Super-Resolution Through Wasserstein GANs
• Few-Shot Adversarial Domain Adaptation
• Learning Implicit Generative Models by Teaching Explicit Ones
• TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation
• Viewmaker Networks: Learning Views for Unsupervised Representation Learning
• Variational f-divergence Minimization
• Image-to-Image Translation: Methods and Applications
• Boundary-Seeking Generative Adversarial Networks
• Deep Learning in Mobile and Wireless Networking: A Survey
• Weighted Support Points from Random Measures: An Interpretable Alternative for Generative Modeling
• Unbalanced GANs: Pre-training the Generator of Generative Adversarial Network using Variational Autoencoder
• Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
• SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Learning the Loss Functions in a Discriminative Space for Video Restoration
• Conditioning of three-dimensional generative adversarial networks for pore and reservoir-scale models
• On the Diversity of Realistic Image Synthesis
• Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN
• Towards an Understanding of Neural Networks in Natural-Image Spaces
• Machine Learning for Reliability Engineering and Safety Applications: Review of Current Status and Future Opportunities
• Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
• System Identification via Validation and Adaptation for Model Updating Applied to a Nonlinear Cantilever Beam
• Adversarial Examples and the Deeper Riddle of Induction: The Need for a Theory of Artifacts in Deep Learning
• Using General Adversarial Networks for Marketing: A Case Study of Airbnb
• Study of Deep Generative Models for Inorganic Chemical Compositions
• The Six Fronts of the Generative Adversarial Networks
• Measuring Fairness in Generative Models
• Predicting Opioid Relapse Using Social Media Data
• Synthetic Data Generators: Sequential and Private
• Generative Adversarial Data Programming
• Wasserstein Learning of Deep Generative Point Process Models
• Generalized Dual Discriminator GANs
• Novelty Detection with GAN
• GANs for Medical Image Analysis
• A Survey on Generative Adversarial Networks: Variants, Applications, and\n Training
• Continual Learning with Adaptive Weights (CLAW)
• DiscrimNet: Semi-Supervised Action Recognition from Videos using Generative Adversarial Networks
• Geometric Entropic Exploration
• Towards a Better Understanding and Regularization of GAN Training Dynamics
• Context-aware Human Intent Inference for Improving Human Machine Cooperation
• Spatiotemporal deep learning models for detection of rapid intensification in cyclones
• ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks
• The Bayesian Approach to Continual Learning: An Overview
• AuthorGAN: Improving GAN Reproducibility using a Modular GAN Framework
• Convergence Problems with Generative Adversarial Networks (GANs)
• Learning to navigate image manifolds induced by generative adversarial networks for unsupervised video generation
• Deep Multi-class Adversarial Specularity Removal
• Adversarial Regression. Generative Adversarial Networks for Non-Linear Regression: Theory and Assessment
• Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation
• Learning Generative Models of Structured Signals from Their Superposition Using GANs with Application to Denoising and Demixing
• Lessons Learned from the Training of GANs on Artificial Datasets
• Structural System Identification via Validation and Adaptation
• Multi-Generator Generative Adversarial Nets
• Relaxed Wasserstein with Applications to GANs
• DeVLearn: A Deep Visual Learning Framework for Localizing Temporary Faults in Power Systems
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Virtual Conditional Generative Adversarial Networks
• Deep generative models as the probability transformation functions
• Active Learning Under Malicious Mislabeling and Poisoning Attacks
• Distilling the Knowledge from Conditional Normalizing Flows
• CopulaSMOTE: A Copula-Based Oversampling Approach for Imbalanced Classification in Diabetes Prediction
• Image Captioning Based on a Hierarchical Attention Mechanism and Policy Gradient Optimization
• Variational Autoencoding the Lagrangian Trajectories of Particles in a Combustion System
• Deep Kernel Bayesian Optimisation for Closed-Loop Electrode Microstructure Design with User-Defined Properties based on GANs
• Learning Class Unique Features in Fine-Grained Visual Classification
• Generative Adversarial Active Learning for Unsupervised Outlier Detection
• Unsupervised Representation Adversarial Learning Network: from Reconstruction to Generation
• Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures
• BIM Hyperreality: Data Synthesis Using BIM and Hyperrealistic Rendering for Deep Learning
• A survey of face recognition techniques under occlusion
• Connecting Touch and Vision via Cross-Modal Prediction
• Word Translation Without Parallel Data
• PBGen: Partial Binarization of Deconvolution-Based Generators for Edge Intelligence
• MMGAN: Manifold Matching Generative Adversarial Network
• Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video
• AE-OT-GAN: Training GANs from data specific latent distribution
• How to Unlock Time Series Editing? Diffusion-Driven Approach with Multi-Grained Control
• Robust Imitation of Diverse Behaviors
• Generating Continuous Representations of Medical Texts
• A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Games
• Non-parametric estimation of Jensen-Shannon Divergence in Generative Adversarial Network training
• Host-Pathongen Co-evolution Inspired Algorithm Enables Robust GAN Training
• ACtuAL: Actor-Critic Under Adversarial Learning
• Can NetGAN be improved on short random walks?
• Dist-GAN: An Improved GAN using Distance Constraints
• Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
• Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
• Lipschitz Generative Adversarial Nets
• Two-stage generative adversarial networks for document image binarization with color noise and background removal
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• GANs May Have No Nash Equilibria
• PointAugment: an Auto-Augmentation Framework for Point Cloud Classification
• Adversarial Learning for Improved Onsets and Frames Music Transcription
• Capturing the diversity of biological tuning curves using generative adversarial networks
• Optimizing the Latent Space of Generative Networks
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Lightweight machine unlearning in neural network
• Puzzle-AE: Novelty Detection in Images through Solving Puzzles
• An Online Learning Approach to Generative Adversarial Networks
• Visual and Semantic Prototypes-Jointly Guided CNN for Generalized Zero-shot Learning
• Behind the Screens: Uncovering Bias in AI-Driven Video Interview Assessments Using Counterfactuals
• Anomaly localization by modeling perceptual features
• Decoupled Learning for Conditional Adversarial Networks
• A Marauder's Map of Security and Privacy in Machine Learning
• Correcting differences in multi-site neuroimaging data using Generative Adversarial Networks
• Hierarchical Modes Exploring in Generative Adversarial Networks
• Automatic Detection of Vague Words and Sentences in Privacy Policies
• Reparameterized Sampling for Generative Adversarial Networks
• On Computation and Generalization of GANs with Spectrum Control
• Rethinking Monocular Depth Estimation with Adversarial Training
• Robust GANs against Dishonest Adversaries
• Domain Partitioning Network
• Conditional Adversarial Generative Flow for Controllable Image Synthesis
• Unsupervised Domain Adaptation through Self-Supervision
• Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation
• WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points
• A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
• Generating Natural Adversarial Examples
• State of the Art on Neural Rendering
• Beyond Sequential Prediction: Learning Financial Market Dynamics in Volatile and Non-Stationary Environments through Sentiment-Conditioned Generative Modelling
• Stochastic Adversarial Video Prediction
• Transductive Zero-Shot Learning by Decoupled Feature Generation
• Pulsar Candidate Identification with Artificial Intelligence Techniques
• Micro-CT Synthesis and Inner Ear Super Resolution via Generative Adversarial Networks and Bayesian Inference
• Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss
• Densely Connected Pyramid Dehazing Network
• Collective Learning Mechanism based Optimal Transport Generative Adversarial Network for Non-parallel Voice Conversion
• Metropolis-Hastings view on variational inference and adversarial training
• Tessellated Wasserstein Auto-Encoders
• Convolutional Normalizing Flows for Deep Gaussian Processes
• Semi-supervised Semantic Segmentation of Prostate and Organs-at-Risk on 3D Pelvic CT Images
• Generative Artificial Intelligence for Internet of Things Computing: A Systematic Survey

Discussions

• NIPS 2016 Tutorial: Generative Adversarial Networks [hn, 4 points, 0 comments]
• NIPS 2016 Tutorial: Generative Adversarial Networks [hn, 3 points, 0 comments]
• DL tutorial by Ian Goodfellow on generative adversarial networks (GANs) [hn, 3 points, 0 comments]
• Generative Adversarial Networks [hn, 3 points, 0 comments]

Related

• Generative adversarial networks
• A Chronological Survey of Theoretical Advancements in Generative Adversarial Networks for Computer Vision
• Generative Adversarial Networks: An Overview
• Generative Adversarial Networks (GANs): What it can generate and What it cannot?
• Generative Adversarial Networks and Other Generative Models