2020/07/30 by He Huang, Huang, He, Yuanwei Chen +12 · 2 citations
Computer Science · Mathematics · #Artificial intelligence #Artificial neural network #Class (philosophy) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Contextual image classification #Domain Adaptation and Few-Shot Learning #Domain knowledge #Exploit #FOS: Computer and information sciences #Generalization #Graph #Image (mathematics) #Knowledge graph #Machine learning #Mathematics #Multimodal Machine Learning Applications #Overfitting #Pattern recognition (psychology) #Text and Document Classification Technologies #Theoretical computer science #Transfer of learning #cs.CV
paper · pdf · doi:10.48550/arxiv.2007.15610
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2020/07/30 · arxiv created 2020/07/31 · arxiv updated 2020/08/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Multi-label zero-shot classification aims to predict multiple unseen class labels for an input image. It is more challenging than its single-label counterpart. On one hand, the unconstrained number of labels assigned to each image makes the model more easily overfit to those seen classes. On the other hand, there is a large semantic gap between seen and unseen classes in the existing multi-label classification datasets. To address these difficult issues, this paper introduces a novel multi-label zero-shot classification framework by learning to transfer from external knowledge. We observe that ImageNet is commonly used to pretrain the feature extractor and has a large and fine-grained label space. This motivates us to exploit it as external knowledge to bridge the seen and unseen classes and promote generalization. Specifically, we construct a knowledge graph including not only classes from the target dataset but also those from ImageNet. Since ImageNet labels are not available in the target dataset, we propose a novel PosVAE module to infer their initial states in the extended knowledge graph. Then we design a relational graph convolutional network (RGCN) to propagate information among classes and achieve knowledge transfer. Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed approach.