Compressive Learning of Generative Networks

2020/02/12 by Vincent Schellekens, Laurent Jacques, Schellekens, Vincent +1
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Neural Networks and Applications #Sparse and Compressive Sensing Techniques #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2002.05095

openalex publication_date 2020/02/12 · arxiv created 2020/03/02 · arxiv updated 2020/03/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Generative networks implicitly approximate complex densities from their sampling with impressive accuracy. However, because of the enormous scale of modern datasets, this training process is often computationally expensive. We cast generative network training into the recent framework of compressive learning: we reduce the computational burden of large-scale datasets by first harshly compressing them in a single pass as a single sketch vector. We then propose a cost function, which approximates the Maximum Mean Discrepancy metric, but requires only this sketch, which makes it time- and memory-efficient to optimize.

Citations

• Generative Moment Matching Networks
• Compressed Sensing using Generative Models
• Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
• Training generative neural networks via Maximum Mean Discrepancy optimization

Related

• How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View
• Generative Moment Matching Networks
• Sketching Datasets for Large-Scale Learning (long version)
• Generative Ratio Matching Networks
• Compression Implies Generalization