Few-Shot Object Detection with Fully Cross-Transformer

2022/03/28 by Guangxing Han, Jiawei Ma, Han, Guangxing +8 · 10 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Domain Adaptation and Few-Shot Learning #Engineering #FOS: Computer and information sciences #Feature extraction #Multimedia (cs.MM) #Multimodal Machine Learning Applications #Object detection #Pascal (unit) #Pattern recognition (psychology) #Single shot #Transformer #Voltage #cs.AI #cs.CV #cs.MM

paper · pdf · doi:10.48550/arxiv.2203.15021

published in arXiv (Cornell University) (Cornell University) · CVPR 2022 (Oral). Code is available at https://github.com/GuangxingHan/FCT

openalex publication_date 2022/03/28 · arxiv created 2022/09/29 · arxiv updated 2022/09/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Few-shot object detection (FSOD), with the aim to detect novel objects using very few training examples, has recently attracted great research interest in the community. Metric-learning based methods have been demonstrated to be effective for this task using a two-branch based siamese network, and calculate the similarity between image regions and few-shot examples for detection. However, in previous works, the interaction between the two branches is only restricted in the detection head, while leaving the remaining hundreds of layers for separate feature extraction. Inspired by the recent work on vision transformers and vision-language transformers, we propose a novel Fully Cross-Transformer based model (FCT) for FSOD by incorporating cross-transformer into both the feature backbone and detection head. The asymmetric-batched cross-attention is proposed to aggregate the key information from the two branches with different batch sizes. Our model can improve the few-shot similarity learning between the two branches by introducing the multi-level interactions. Comprehensive experiments on both PASCAL VOC and MSCOCO FSOD benchmarks demonstrate the effectiveness of our model.

Cited by

• Decoupling Template Bias in CLIP: Harnessing Empty Prompts for Enhanced Few-Shot Learning
• FSP-DETR: Few-Shot Prototypical Parasitic Ova Detection
• Few-Shot Pattern Detection via Template Matching and Regression
• Few-Shot Object Detection via Spatial-Channel State Space Model
• Few-Shot Learning in Video and 3D Object Detection: A Survey
• Fine-Grained Zero-Shot Object Detection
• No time to train! Training-Free Reference-Based Instance Segmentation
• Visual Textualization for Image Prompted Object Detection
• Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation
• CDFormer: Cross-Domain Few-Shot Object Detection Transformer Against Feature Confusion

Related

• Few-Shot Object Detection: Research Advances and Challenges
• Recent Few-Shot Object Detection Algorithms: A Survey with Performance Comparison
• Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detector
• Beyond Few-shot Object Detection: A Detailed Survey
• Siamese Transformer Networks for Few-shot Image Classification