2022/10/14 by Hyeong Kyu Choi, Choi, Hyeong Kyu, Joonmyung Choi +3 · 3 citations
Computer Science · Engineering · Medicine · #Advanced Neural Network Applications #Advanced X-ray and CT Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Medical Imaging Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2210.07562
openalex publication_date 2022/10/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Mixup is a commonly adopted data augmentation technique for image classification. Recent advances in mixup methods primarily focus on mixing based on saliency. However, many saliency detectors require intense computation and are especially burdensome for parameter-heavy transformer models. To this end, we propose TokenMixup, an efficient attention-guided token-level data augmentation method that aims to maximize the saliency of a mixed set of tokens. TokenMixup provides x15 faster saliency-aware data augmentation compared to gradient-based methods. Moreover, we introduce a variant of TokenMixup which mixes tokens within a single instance, thereby enabling multi-scale feature augmentation. Experiments show that our methods significantly improve the baseline models' performance on CIFAR and ImageNet-1K, while being more efficient than previous methods. We also reach state-of-the-art performance on CIFAR-100 among from-scratch transformer models. Code is available at https://github.com/mlvlab/TokenMixup.