Generative Moment Matching Networks

2015/02/10 by Yujia Li, Kevin Swersky, Li, Yujia +3 · 28 citations
Computer Science · #Artificial Intelligence (cs.AI) #Digital Media Forensic Detection #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing

paper · pdf · doi:10.48550/arxiv.1502.02761

openalex publication_date 2015/02/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of learning deep generative models from data. We formulate a method that generates an independent sample via a single feedforward pass through a multilayer perceptron, as in the recently proposed generative adversarial networks (Goodfellow et al., 2014). Training a generative adversarial network, however, requires careful optimization of a difficult minimax program. Instead, we utilize a technique from statistical hypothesis testing known as maximum mean discrepancy (MMD), which leads to a simple objective that can be interpreted as matching all orders of statistics between a dataset and samples from the model, and can be trained by backpropagation. We further boost the performance of this approach by combining our generative network with an auto-encoder network, using MMD to learn to generate codes that can then be decoded to produce samples. We show that the combination of these techniques yields excellent generative models compared to baseline approaches as measured on MNIST and the Toronto Face Database.

Citations

• Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
• Input Warping for Bayesian Optimization of Non-stationary Functions
• Improving neural networks by preventing co-adaptation of feature detectors
• Auto-Encoding Variational Bayes
• Connectionist learning of belief networks
• Gradient-based learning applied to document recognition
• Training Products of Experts by Minimizing Contrastive Divergence
• Deep Generative Stochastic Networks Trainable by Backprop
• Sequence to Sequence Learning with Neural Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• Generalized Denoising Auto-Encoders as Generative Models
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups

Cited by

• Minimum Distance Summaries for Robust Neural Posterior Estimation
• Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD
• Microstructure-based Variational Neural Networks for Robust Uncertainty Quantification in Materials Digital Twins
• Maximum Mean Discrepancy with Unequal Sample Sizes via Generalized U-Statistics
• Fusion of classical and quantum kernels enables accurate and robust two-sample tests
• SynQuE: Estimating Synthetic Dataset Quality Without Annotations
• Amortized Moment Matching for Visual Generation
• Generative Modeling via Drifting
• Improved Training Technique for Shortcut Models
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• FnRGNN: Distribution-aware Fairness in Graph Neural Network
• PolyGraph Discrepancy: a classifier-based metric for graph generation
• One-shot Conditional Sampling: MMD meets Nearest Neighbors
• Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm
• ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations
• Trainability and Mode Separation of Mixed IQP-QCBMs
• Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
• Cross-Modal Retrieval with Cauchy-Schwarz Divergence
• E-ROBOT: a dimension-free method for robust statistics and machine learning via Schrödinger bridge
• Learning Majority-to-Minority Transformations with MMD and Triplet Loss for Imbalanced Classification
• Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibration
• Divergence-kernel method for linear responses of densities and generative models
• Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance
• Adaptive generative moment matching networks for improved learning of dependence structures
• Quantum-inspired probability metrics define a complete, universal space for statistical learning
• Distribution Matching via Generalized Consistency Models
• Learning Kernel Tests Without Data Splitting
• GANs with Conditional Independence Graphs: On Subadditivity of Probability Divergences

Related

• Conditional Generative Moment-Matching Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Generative Ratio Matching Networks
• Generative Adversarial Networks
• Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy