2020/11/20 by Alireza Zareian, Kevin Dela Rosa, Zareian, Alireza +6 · 61 citations
Computer Science · #Advanced Neural Network Applications #Annotation #Artificial intelligence #Artificial neural network #Bounding overwatch #Computer science #Computer vision #Domain Adaptation and Few-Shot Learning #Image (mathematics) #Machine learning #Minimum bounding box #Multimodal Machine Learning Applications #Object (grammar) #Object detection #Pattern recognition (psychology) #Scalability #Scale (ratio) #Set (abstract data type) #Supervised learning #Vocabulary #cs.AI #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2011.10678
published in arXiv (Cornell University) (Cornell University) · To be presented at CVPR 2021 (oral paper)
openalex publication_date 2020/11/20 · arxiv created 2021/03/14 · arxiv updated 2021/03/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Despite the remarkable accuracy of deep neural networks in object detection, they are costly to train and scale due to supervision requirements. Particularly, learning more object categories typically requires proportionally more bounding box annotations. Weakly supervised and zero-shot learning techniques have been explored to scale object detectors to more categories with less supervision, but they have not been as successful and widely adopted as supervised models. In this paper, we put forth a novel formulation of the object detection problem, namely open-vocabulary object detection, which is more general, more practical, and more effective than weakly supervised and zero-shot approaches. We propose a new method to train object detectors using bounding box annotations for a limited set of object categories, as well as image-caption pairs that cover a larger variety of objects at a significantly lower cost. We show that the proposed method can detect and localize objects for which no bounding box annotation is provided during training, at a significantly higher accuracy than zero-shot approaches. Meanwhile, objects with bounding box annotation can be detected almost as accurately as supervised methods, which is significantly better than weakly supervised baselines. Accordingly, we establish a new state of the art for scalable object detection.