Meta Approach to Data Augmentation Optimization

2020/06/14 by Ryuichiro Hataya, Hataya, Ryuichiro, Jan Zdenek +5 · 3 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2006.07965

openalex publication_date 2020/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Data augmentation policies drastically improve the performance of image recognition tasks, especially when the policies are optimized for the target data and tasks. In this paper, we propose to optimize image recognition models and data augmentation policies simultaneously to improve the performance using gradient descent. Unlike prior methods, our approach avoids using proxy tasks or reducing search space, and can directly improve the validation performance. Our method achieves efficient and scalable training by approximating the gradient of policies by implicit gradient with Neumann series approximation. We demonstrate that our approach can improve the performance of various image classification tasks, including ImageNet classification and fine-grained recognition, without using dataset-specific hyperparameter tuning.

Citations

• Gradient-based Hyperparameter Optimization through Reversible Learning
• ImageNet Large Scale Visual Recognition Challenge
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Hyperparameter optimization with approximate gradient
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Improved Regularization of Convolutional Neural Networks with Cutout
• mixup: Beyond Empirical Risk Minimization
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian\n Augmentation
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Momentum Contrast for Unsupervised Visual Representation Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Learning Multiple Layers of Features from Tiny Images

Cited by

• On-the-Fly Data Augmentation via Gradient-Guided and Sample-Aware Influence Estimation
• Robust Bioacoustic Detection via Richly Labelled Synthetic Soundscape Augmentation
• Teaching with Commentaries

Related

• MetaAugment: Sample-Aware Data Augmentation Policy Learning
• Learning Data Augmentation with Online Bilevel Optimization for Image Classification
• AutoAugment: Learning Augmentation Policies from Data
• Learning Optimal Data Augmentation Policies via Bayesian Optimization for Image Classification Tasks
• No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets