2023/05/02 by George Cazenavette, Cazenavette, George, Tongzhou Wang +7 · 18 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computational Physics and Python Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image Processing and 3D Reconstruction #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2305.01649
openalex publication_date 2023/05/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Dataset Distillation aims to distill an entire dataset's knowledge into a few synthetic images. The idea is to synthesize a small number of synthetic data points that, when given to a learning algorithm as training data, result in a model approximating one trained on the original data. Despite recent progress in the field, existing dataset distillation methods fail to generalize to new architectures and scale to high-resolution datasets. To overcome the above issues, we propose to use the learned prior from pre-trained deep generative models to synthesize the distilled data. To achieve this, we present a new optimization algorithm that distills a large number of images into a few intermediate feature vectors in the generative model's latent space. Our method augments existing techniques, significantly improving cross-architecture generalization in all settings.