ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation

2025/07/12 by Abdulvahap Mutlu, Şengül Doğan, Mutlu, Abdulvahap +3
Computer Science · Engineering · #Advanced Image and Video Retrieval Techniques #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Processing Techniques and Applications #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2507.09299

openalex publication_date 2025/07/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The remarkable representational power of Vision Transformers (ViTs) remains underutilized in few-shot image classification. In this work, we introduce ViT-ProtoNet, which integrates a ViT-Small backbone into the Prototypical Network framework. By averaging class conditional token embeddings from a handful of support examples, ViT-ProtoNet constructs robust prototypes that generalize to novel categories under 5-shot settings. We conduct an extensive empirical evaluation on four standard benchmarks: Mini-ImageNet, FC100, CUB-200, and CIFAR-FS, including overlapped support variants to assess robustness. Across all splits, ViT-ProtoNet consistently outperforms CNN-based prototypical counterparts, achieving up to a 3.2% improvement in 5-shot accuracy and demonstrating superior feature separability in latent space. Furthermore, it outperforms or is competitive with transformer-based competitors using a more lightweight backbone. Comprehensive ablations examine the impact of transformer depth, patch size, and fine-tuning strategy. To foster reproducibility, we release code and pretrained weights. Our results establish ViT-ProtoNet as a powerful, flexible approach for few-shot classification and set a new baseline for transformer-based meta-learners.

Citations

• The Balanced-Pairwise-Affinities Feature Transform
• Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification
• Rethinking Generalization in Few-Shot Classification
• Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference
• Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• CrossTransformers: spatially-aware few-shot transfer
• Laplacian Regularized Few-Shot Learning
• Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning
• Task Augmentation by Rotating for Meta-Learning
• Edge-labeling Graph Neural Network for Few-shot Learning
• A Closer Look at Few-shot Classification
• Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Meta-learning with differentiable closed-form solvers
• On First-Order Meta-Learning Algorithms
• Learning to Compare: Relation Network for Few-Shot Learning
• Few-Shot Learning with Graph Neural Networks
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• Attention Is All You Need
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Matching Networks for One Shot Learning
• Deep Residual Learning for Image Recognition
• Gradient-based learning applied to document recognition

Related

• Siamese Transformer Networks for Few-shot Image Classification
• Strong Baselines for Parameter Efficient Few-Shot Fine-tuning
• Vision Transformers in 2022: An Update on Tiny ImageNet
• Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference
• Dense Vision Transformer Compression with Few Samples