PolyTransform: Deep Polygon Transformer for Instance Segmentation

2019/12/05 by Justin Liang, Liang, Justin, Namdar Homayounfar +9 · 4 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Retrieval and Classification Techniques

paper · pdf · doi:10.48550/arxiv.1912.02801

openalex publication_date 2019/12/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods. In particular, we first exploit a segmentation network to generate instance masks. We then convert the masks into a set of polygons that are then fed to a deforming network that transforms the polygons such that they better fit the object boundaries. Our experiments on the challenging Cityscapes dataset show that our PolyTransform significantly improves the performance of the backbone instance segmentation network and ranks 1st on the Cityscapes test-set leaderboard. We also show impressive gains in the interactive annotation setting. We release the code at https://github.com/uber-research/PolyTransform.

Citations

• Adam: A Method for Stochastic Optimization
• Associative Embedding: End-to-End Learning for Joint Detection and\n Grouping
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal\n Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Hybrid Task Cascade for Instance Segmentation
• Fast Interactive Object Annotation with Curve-GCN
• DARNet: Deep Active Ray Network for Building Segmentation
• Feature Pyramid Networks for Object Detection
• Deep Residual Learning for Image Recognition
• Deformable Convolutional Networks
• Deep Extreme Cut: From Extreme Points to Object Segmentation
• An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

Cited by

• HoliTracer: Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery
• Unified Medical Image Segmentation with State Space Modeling Snake
• MARL-MambaContour: Unleashing Multi-Agent Deep Reinforcement Learning for Active Contour Optimization in Medical Image Segmentation
• Uncertainty-Masked Bernoulli Diffusion for Camouflaged Object Detection Refinement

Related

• BoxSnake: Polygonal Instance Segmentation with Box Supervision
• Deep Watershed Transform for Instance Segmentation
• PolyFormer: Referring Image Segmentation as Sequential Polygon Generation
• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation
• CenterPoly: real-time instance segmentation using bounding polygons