Towards Open World Object Detection

2021/03/03 by K J Joseph, Salman Khan, Joseph, K J +5 · 30 citations
Computer Science · Mathematics · Psychology · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Curiosity #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Forgetting #Human–computer interaction #Identification (biology) #Instinct #Machine Learning (cs.LG) #Machine learning #Mathematics #Multimodal Machine Learning Applications #Object (grammar) #Object detection #Open research #Pattern recognition (psychology) #Product (mathematics) #Protocol (science) #Psychology #World Wide Web #cs.AI #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2103.02603

published in Research Archive of Indian Institute of Technology Hyderabad (Indian Institute of Technology Hyderabad) (Indian Institute of Technology Hyderabad) · To appear in CVPR 2021 as an ORAL paper. Code is available in https://github.com/JosephKJ/OWOD

openalex publication_date 2021/03/03 · arxiv created 2021/05/09 · arxiv updated 2021/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: `Open World Object Detection', where a model is tasked to: 1) identify objects that have not been introduced to it as `unknown', without explicit supervision to do so, and 2) incrementally learn these identified unknown categories without forgetting previously learned classes, when the corresponding labels are progressively received. We formulate the problem, introduce a strong evaluation protocol and provide a novel solution, which we call ORE: Open World Object Detector, based on contrastive clustering and energy based unknown identification. Our experimental evaluation and ablation studies analyze the efficacy of ORE in achieving Open World objectives. As an interesting by-product, we find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting, where we achieve state-of-the-art performance, with no extra methodological effort. We hope that our work will attract further research into this newly identified, yet crucial research direction.

Citations

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• The Pascal Visual Object Classes (VOC) Challenge
• Learning without Forgetting
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Optimal Continual Learning has Perfect Memory and is NP-hard

Cited by

• CountGD++: Generalized Prompting for Open-World Counting
• Multimodal Semantic-Probabilistic Objectness for Open World Object Detection
• Fast Post-Hoc Confidence Fusion for 3-Class Open-Set Aerial Object Detection
• TSRE: Channel-Aware Typical Set Refinement for Out-of-Distribution Detection
• Online Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Network
• Towards 3D Objectness Learning in an Open World
• TARO: Toward Semantically Rich Open-World Object Detection
• Looking Beyond the Known: Towards a Data Discovery Guided Open-World Object Detection
• C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detection
• MVT: Mask-Grounded Vision-Language Models for Taxonomy-Aligned Land-Cover Tagging
• Towards Open World Detection: A Survey
• OASIS: Open-world Adaptive Self-supervised and Imbalanced-aware System
• Index-Aligned Query Distillation for Transformer-based Incremental Object Detection
• OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration
• Decoupled PROB: Decoupled Query Initialization Tasks and Objectness-Class Learning for Open World Object Detection
• Hierarchical Neural Collapse Detection Transformer for Class Incremental Object Detection
• ADAM: Autonomous Discovery and Annotation Model using LLMs for Context-Aware Annotations
• Consistent Supervised-Unsupervised Alignment for Generalized Category Discovery
• Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach
• OoDDINO:A Multi-level Framework for Anomaly Segmentation on Complex Road Scenes
• OBSER: Object-Based Sub-Environment Recognition for Zero-Shot Environmental Inference
• Open-World Object Counting in Videos
• FindMeIfYouCan: Bringing Open Set metrics to near , far and farther Out-of-Distribution Object Detection
• A Mathematical Perspective On Contrastive Learning
• Open-Det: An Efficient Learning Framework for Open-Ended Detection
• MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models
• AHC: Meta-Learned Adaptive Compression for Continual Object Detection on Memory-Constrained Microcontrollers
• OpenAVS: Training-Free Open-Vocabulary Audio Visual Segmentation with Foundational Models
• EffOWT: Transfer Visual Language Models to Open-World Tracking Efficiently and Effectively
• Balancing Stability and Plasticity in Pretrained Detector: A Dual-Path Framework for Incremental Object Detection

Related

• OW-DETR: Open-world Detection Transformer
• Unsupervised Recognition of Unknown Objects for Open-World Object Detection
• Addressing the Challenges of Open-World Object Detection
• OW-Rep: Open World Object Detection with Instance Representation Learning
• UC-OWOD: Unknown-Classified Open World Object Detection