A Universal Representation Transformer Layer for Few-Shot Image Classification

2020/06/21 by Lu Liu, William L. Hamilton, Liu, Lu +7 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Advanced Neural Network Applications #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.2006.11702

Abstract

Few-shot classification aims to recognize unseen classes when presented with only a small number of samples. We consider the problem of multi-domain few-shot image classification, where unseen classes and examples come from diverse data sources. This problem has seen growing interest and has inspired the development of benchmarks such as Meta-Dataset. A key challenge in this multi-domain setting is to effectively integrate the feature representations from the diverse set of training domains. Here, we propose a Universal Representation Transformer (URT) layer, that meta-learns to leverage universal features for few-shot classification by dynamically re-weighting and composing the most appropriate domain-specific representations. In experiments, we show that URT sets a new state-of-the-art result on Meta-Dataset. Specifically, it achieves top-performance on the highest number of data sources compared to competing methods. We analyze variants of URT and present a visualization of the attention score heatmaps that sheds light on how the model performs cross-domain generalization. Our code is available at https://github.com/liulu112601/URT.

Citations

• Gradient-based learning applied to document recognition
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Universal representations:The missing link between faces, text, planktons, and cat breeds

Cited by

• Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models
• Transformers in Vision: A Survey

Related

• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification
• Universal Representation Learning from Multiple Domains for Few-shot Classification
• Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification
• Few-Shot Classification in Unseen Domains by Episodic Meta-Learning Across Visual Domains
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation