Learning What Not to Segment: A New Perspective on Few-Shot Segmentation

2022/03/15 by Chunbo Lang, Gong Cheng, Lang, Chunbo +5 · 15 citations
Computer Science · Mathematics · Medicine · #Advanced Neural Network Applications #Artificial intelligence #Base (topology) #COVID-19 diagnosis using AI #Class (philosophy) #Code (set theory) #Computer science #Domain Adaptation and Few-Shot Learning #Generalization #Machine learning #Mathematics #Meta learning (computer science) #Pascal (unit) #Pattern recognition (psychology) #Perspective (graphical) #Pixel #Programming language #Scheme (mathematics) #Segmentation #Task (project management) #cs.CV

paper · pdf · doi:10.48550/arxiv.2203.07615

published in arXiv (Cornell University) (Cornell University) · Accepted to CVPR 2022 Oral

openalex publication_date 2022/03/15 · arxiv created 2022/03/29 · arxiv updated 2022/03/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recently few-shot segmentation (FSS) has been extensively developed. Most previous works strive to achieve generalization through the meta-learning framework derived from classification tasks; however, the trained models are biased towards the seen classes instead of being ideally class-agnostic, thus hindering the recognition of new concepts. This paper proposes a fresh and straightforward insight to alleviate the problem. Specifically, we apply an additional branch (base learner) to the conventional FSS model (meta learner) to explicitly identify the targets of base classes, i.e., the regions that do not need to be segmented. Then, the coarse results output by these two learners in parallel are adaptively integrated to yield precise segmentation prediction. Considering the sensitivity of meta learner, we further introduce an adjustment factor to estimate the scene differences between the input image pairs for facilitating the model ensemble forecasting. The substantial performance gains on PASCAL-5i and COCO-20i verify the effectiveness, and surprisingly, our versatile scheme sets a new state-of-the-art even with two plain learners. Moreover, in light of the unique nature of the proposed approach, we also extend it to a more realistic but challenging setting, i.e., generalized FSS, where the pixels of both base and novel classes are required to be determined. The source code is available at github.com/chunbolang/BAM.

Cited by

• Take a Peek: Efficient Encoder Adaptation for Few-Shot Semantic Segmentation via LoRA
• DistillFSS: Synthesizing Few-Shot Knowledge into a Lightweight Segmentation Model
• Matching-Based Few-Shot Semantic Segmentation Models Are Interpretable by Design
• Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentation
• Unbiased Semantic Decoding with Vision Foundation Models for Few-shot Segmentation
• Object-level Correlation for Few-Shot Segmentation
• Correspondence as Video: Test-Time Adaption on SAM2 for Reference Segmentation in the Wild
• Balancing Conservatism and Aggressiveness: Prototype-Affinity Hybrid Network for Few-Shot Segmentation
• No time to train! Training-Free Reference-Based Instance Segmentation
• Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation
• ProSAM: Enhancing the Robustness of SAM-based Visual Reference Segmentation with Probabilistic Prompts
• SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentation
• INSID3: Training-Free In-Context Segmentation with DINOv3
• UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelation
• DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency
• Learning Attribute-aware Representations for Few-shot Scene Text Segmentation

Related

• Elimination of Non-Novel Segments at Multi-Scale for Few-Shot Segmentation
• Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement
• Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes
• Mining Latent Classes for Few-shot Segmentation
• Learning Meta-class Memory for Few-Shot Semantic Segmentation