2023/05/22 by Emirhan Kurtuluş, Zichao Li, Kurtulus, Emirhan +5 · 1 citation
Computer Science · Medicine · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Augment #COVID-19 diagnosis using AI #Code (set theory) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generalization #Latency (audio) #Machine Learning (cs.LG) #Machine learning #Representation (politics) #Similarity (geometry) #Source code
paper · pdf · doi:10.48550/arxiv.2305.13520
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/05/22 · openalex created_date 2023/05/26 · openalex updated_date 2026/07/28
Data augmentation methods have played an important role in the recent advance of deep learning models, and have become an indispensable component of state-of-the-art models in semi-supervised, self-supervised, and supervised training for vision. Despite incurring no additional latency at test time, data augmentation often requires more epochs of training to be effective. For example, even the simple flips-and-crops augmentation requires training for more than 5 epochs to improve performance, whereas RandAugment requires more than 90 epochs. We propose a general framework called Tied-Augment, which improves the efficacy of data augmentation in a wide range of applications by adding a simple term to the loss that can control the similarity of representations under distortions. Tied-Augment can improve state-of-the-art methods from data augmentation (e.g. RandAugment, mixup), optimization (e.g. SAM), and semi-supervised learning (e.g. FixMatch). For example, Tied-RandAugment can outperform RandAugment by 2.0% on ImageNet. Notably, using Tied-Augment, data augmentation can be made to improve generalization even when training for a few epochs and when fine-tuning. We open source our code at https://github.com/ekurtulus/tied-augment/tree/main.