Semi-convolutional Operators for Instance Segmentation

2018/07/27 by David Novotný, David Novotny, Novotny, David +6 · 8 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial intelligence #Bounding overwatch #Cell Image Analysis Techniques #Classifier (UML) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Convolutional neural network #FOS: Computer and information sciences #Hough transform #Image (mathematics) #Kernel (algebra) #Mathematics #Object detection #Pascal (unit) #Pattern recognition (psychology) #Pixel #Segmentation #cs.CV

paper · pdf · doi:10.48550/arxiv.1807.10712

published in arXiv (Cornell University) (Cornell University) · Accepted as a conference paper at ECCV 2018

arxiv created 2018/07/27 · openalex publication_date 2018/07/27 · arxiv updated 2018/07/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Object detection and instance segmentation are dominated by region-based methods such as Mask RCNN. However, there is a growing interest in reducing these problems to pixel labeling tasks, as the latter could be more efficient, could be integrated seamlessly in image-to-image network architectures as used in many other tasks, and could be more accurate for objects that are not well approximated by bounding boxes. In this paper we show theoretically and empirically that constructing dense pixel embeddings that can separate object instances cannot be easily achieved using convolutional operators. At the same time, we show that simple modifications, which we call semi-convolutional, have a much better chance of succeeding at this task. We use the latter to show a connection to Hough voting as well as to a variant of the bilateral kernel that is spatially steered by a convolutional network. We demonstrate that these operators can also be used to improve approaches such as Mask RCNN, demonstrating better segmentation of complex biological shapes and PASCAL VOC categories than achievable by Mask RCNN alone.

Citations

• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal\n Networks
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
• Pixel-level Encoding and Depth Layering for Instance-level Semantic Labeling
• Monocular Object Instance Segmentation and Depth Ordering with CNNs
• An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

Cited by

• TextMountain: Accurate Scene Text Detection via Instance Segmentation
• Learning Gaussian Instance Segmentation in Point Clouds

Related

• From Image-Level to Pixel-Level Labeling with Convolutional Networks
• Fully Convolutional Networks for Semantic Segmentation
• Simultaneous Detection and Segmentation
• Segmentation-Aware Convolutional Networks Using Local Attention Masks
• Fully Convolutional Networks for Semantic Segmentation