An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems

2023/05/02 by Weixuan Sun, Zheyuan Liu, Sun, Weixuan +7 · 1 citation
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning and Data Classification #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2305.01586

openalex publication_date 2023/05/02 · openalex created_date 2023/05/05 · openalex updated_date 2026/07/28

Abstract

The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the application of SAM in Weakly-Supervised Semantic Segmentation (WSSS). Particularly, we adapt SAM as the pseudo-label generation pipeline given only the image-level class labels. While we observed impressive results in most cases, we also identify certain limitations. Our study includes performance evaluations on PASCAL VOC and MS-COCO, where we achieved remarkable improvements over the latest state-of-the-art methods on both datasets. We anticipate that this report encourages further explorations of adopting SAM in WSSS, as well as wider real-world applications.

Cited by

• Depth Edge Alignment Loss: DEALing with Depth in Weakly Supervised Semantic Segmentation

Related

• Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation
• Segment Anything is A Good Pseudo-label Generator for Weakly Supervised Semantic Segmentation
• Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Models
• Enhancing Weakly Supervised Semantic Segmentation with Multi-modal Foundation Models: An End-to-End Approach
• Foundation Model Assisted Weakly Supervised Semantic Segmentation