2022/05/30 by Jang-Hyun Kim, Jinuk Kim, Kim, Jang-Hyun +14 · 35 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Speech Recognition and Synthesis #cs.LG
paper · pdf · doi:10.48550/arxiv.2205.14959
ICML 2022; Codes at https://github.com/snu-mllab/Efficient-Dataset-Condensation.git
arxiv created 2022/06/02 · arxiv updated 2022/06/03
The great success of machine learning with massive amounts of data comes at a price of huge computation costs and storage for training and tuning. Recent studies on dataset condensation attempt to reduce the dependence on such massive data by synthesizing a compact training dataset. However, the existing approaches have fundamental limitations in optimization due to the limited representability of synthetic datasets without considering any data regularity characteristics. To this end, we propose a novel condensation framework that generates multiple synthetic data with a limited storage budget via efficient parameterization considering data regularity. We further analyze the shortcomings of the existing gradient matching-based condensation methods and develop an effective optimization technique for improving the condensation of training data information. We propose a unified algorithm that drastically improves the quality of condensed data against the current state-of-the-art on CIFAR-10, ImageNet, and Speech Commands.