Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy

2016/11/14 by Danica J. Sutherland, Hsiao-Yu Fish Tung, Sutherland, Danica J. +11 · 11 citations
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Methods and Mixture Models #Digital Media Forensic Detection #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1611.04488

openalex publication_date 2016/11/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a method to optimize the representation and distinguishability of samples from two probability distributions, by maximizing the estimated power of a statistical test based on the maximum mean discrepancy (MMD). This optimized MMD is applied to the setting of unsupervised learning by generative adversarial networks (GAN), in which a model attempts to generate realistic samples, and a discriminator attempts to tell these apart from data samples. In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples. Second, the MMD can be used to evaluate the performance of a generative model, by testing the model's samples against a reference data set. In the latter role, the optimized MMD is particularly helpful, as it gives an interpretable indication of how the model and data distributions differ, even in cases where individual model samples are not easily distinguished either by eye or by classifier.

Cited by

• Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD
• On the Hardness of Conditional Independence Testing In Practice
• Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics
• Shallow IQP circuits and graph generation
• DUAL: Learning Diverse Kernels for Aggregated Two-sample and Independence Testing
• Anchor-based Maximum Discrepancy for Relative Similarity Testing
• One-shot Conditional Sampling: MMD meets Nearest Neighbors
• Adaptive generative moment matching networks for improved learning of dependence structures
• Learning Kernel Tests Without Data Splitting
• Likelihood Ratio Tests by Kernel Gaussian Embedding
• Revealing posturographic features associated with the risk of falling in\n patients with Parkinsonian syndromes via machine learning

Related

• Training generative neural networks via Maximum Mean Discrepancy optimization
• Demystifying MMD GANs
• Generative Moment Matching Networks
• On the Optimization Landscape of Maximum Mean Discrepancy
• A Brief Introduction to Generative Models