One-Shot Instance Segmentation

2018/11/28 by Claudio Michaelis, Michaelis, Claudio, Ivan Ustyuzhaninov +5 · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1811.11507

openalex publication_date 2018/11/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We tackle the problem of one-shot instance segmentation: Given an example image of a novel, previously unknown object category, find and segment all objects of this category within a complex scene. To address this challenging new task, we propose Siamese Mask R-CNN. It extends Mask R-CNN by a Siamese backbone encoding both reference image and scene, allowing it to target detection and segmentation towards the reference category. We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object categories works very well, targeting the detection network towards the reference category appears to be more difficult. Our work provides a first strong baseline for one-shot instance segmentation and will hopefully inspire further research into more powerful and flexible scene analysis algorithms. Code is available at: https://github.com/bethgelab/siamese-mask-rcnn

Citations

• The Pascal Visual Object Classes (VOC) Challenge
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal\n Networks
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• YOLOv3: An Incremental Improvement
• Large Margin Few-Shot Learning
• Comparison Network for One-Shot Conditional Object Detection
• The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation

Cited by

• Indirect Attention: Turning Context Misalignment into a Feature

Related

• Mask Encoding for Single Shot Instance Segmentation
• Mask R-CNN
• SOLO: Segmenting Objects by Locations
• Learning to Segment Every Thing
• SOLO: A Simple Framework for Instance Segmentation