2021/07/13 by Bowen Cheng, Cheng, Bowen, Alexander G. Schwing +3 · 167 citations
Computer Science · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial intelligence #Binary classification #Binary number #Class (philosophy) #Computer science #Contextual image classification #Domain Adaptation and Few-Shot Learning #Image (mathematics) #Image segmentation #Key (lock) #Machine learning #Mathematics #Pattern recognition (psychology) #Pixel #Segmentation #Semantics (computer science) #Set (abstract data type) #Support vector machine #Task (project management) #cs.CV
paper · pdf · doi:10.48550/arxiv.2107.06278
published in arXiv (Cornell University) (Cornell University) · NeurIPS 2021, Spotlight. Project page: https://bowenc0221.github.io/maskformer
openalex publication_date 2021/07/13 · arxiv created 2021/10/31 · arxiv updated 2021/11/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
Modern approaches typically formulate semantic segmentation as a per-pixel classification task, while instance-level segmentation is handled with an alternative mask classification. Our key insight: mask classification is sufficiently general to solve both semantic- and instance-level segmentation tasks in a unified manner using the exact same model, loss, and training procedure. Following this observation, we propose MaskFormer, a simple mask classification model which predicts a set of binary masks, each associated with a single global class label prediction. Overall, the proposed mask classification-based method simplifies the landscape of effective approaches to semantic and panoptic segmentation tasks and shows excellent empirical results. In particular, we observe that MaskFormer outperforms per-pixel classification baselines when the number of classes is large. Our mask classification-based method outperforms both current state-of-the-art semantic (55.6 mIoU on ADE20K) and panoptic segmentation (52.7 PQ on COCO) models.