Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

2020/10/06 by Yannick Le Cacheux, Cacheux, Yannick Le, Hervé Le Borgne +3
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimedia (cs.MM) #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2010.02959

openalex publication_date 2020/10/06 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

Zero-shot learning aims to recognize instances of unseen classes, for which no visual instance is available during training, by learning multimodal relations between samples from seen classes and corresponding class semantic representations. These class representations usually consist of either attributes, which do not scale well to large datasets, or word embeddings, which lead to poorer performance. A good trade-off could be to employ short sentences in natural language as class descriptions. We explore different solutions to use such short descriptions in a ZSL setting and show that while simple methods cannot achieve very good results with sentences alone, a combination of usual word embeddings and sentences can significantly outperform current state-of-the-art.

Citations

• Zero-Shot Learning by Convex Combination of Semantic Embeddings
• Distributed Representations of Words and Phrases and their Compositionality
• Word Embeddings: A Survey
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

Related

• Transductive Multi-class and Multi-label Zero-shot Learning
• Learning Semantic Ambiguities for Zero-Shot Learning
• Rich Semantics Improve Few-shot Learning
• Train Once, Test Anywhere: Zero-Shot Learning for Text Classification
• Interpretable Zero-shot Learning with Infinite Class Concepts