2021/10/11 by Durvesh Malpure, Malpure, Durvesh, Onkar Litake +4 · 3 citations
Computer Science · Engineering · Mathematics · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Artificial intelligence #Block (permutation group theory) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Domain Adaptation and Few-Shot Learning #Electrical engineering #Engineering #FOS: Computer and information sciences #Mathematics #Transfer of learning #Transformer #Voltage #cs.AI #cs.CV
paper · pdf · doi:10.48550/arxiv.2110.05270
published in arXiv (Cornell University) (Cornell University) · 8 pages, 4 figures
arxiv created 2021/10/11 · openalex publication_date 2021/10/11 · arxiv updated 2021/10/12 · openalex created_date 2021/10/25 · openalex updated_date 2026/07/28
In recent developments in the field of Computer Vision, a rise is seen in the use of transformer-based architectures. They are surpassing the state-of-the-art set by CNN architectures in accuracy but on the other hand, they are computationally very expensive to train from scratch. As these models are quite recent in the Computer Vision field, there is a need to study it's transfer learning capabilities and compare it with CNNs so that we can understand which architecture is better when applied to real world problems with small data. In this work, we follow a simple yet restrictive method for fine-tuning both CNN and Transformer models pretrained on ImageNet1K on CIFAR-10 and compare them with each other. We only unfreeze the last transformer/encoder or last convolutional block of a model and freeze all the layers before it while adding a simple MLP at the end for classification. This simple modification lets us use the raw learned weights of both these neural networks. From our experiments, we find out that transformers-based architectures not only achieve higher accuracy than CNNs but some transformers even achieve this feat with around 4 times lesser number of parameters.