Weakly Supervised Semantic Segmentation using Out-of-Distribution Data

2022/03/08 by Jungbeom Lee, Lee, Jungbeom, Seong Joon Oh +10 · 5 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #cs.CV

paper · pdf · doi:10.48550/arxiv.2203.03860

CVPR 2022

arxiv created 2022/03/08 · openalex publication_date 2022/03/08 · arxiv updated 2022/03/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Weakly supervised semantic segmentation (WSSS) methods are often built on pixel-level localization maps obtained from a classifier. However, training on class labels only, classifiers suffer from the spurious correlation between foreground and background cues (e.g. train and rail), fundamentally bounding the performance of WSSS. There have been previous endeavors to address this issue with additional supervision. We propose a novel source of information to distinguish foreground from the background: Out-of-Distribution (OoD) data, or images devoid of foreground object classes. In particular, we utilize the hard OoDs that the classifier is likely to make false-positive predictions. These samples typically carry key visual features on the background (e.g. rail) that the classifiers often confuse as foreground (e.g. train), so these cues let classifiers correctly suppress spurious background cues. Acquiring such hard OoDs does not require an extensive amount of annotation efforts; it only incurs a few additional image-level labeling costs on top of the original efforts to collect class labels. We propose a method, W-OoD, for utilizing the hard OoDs. W-OoD achieves state-of-the-art performance on Pascal VOC 2012.

Cited by

• SSR: Semantic and Spatial Rectification for CLIP-based Weakly Supervised Segmentation
• Advancing Weakly-Supervised Change Detection in Satellite Images via Adversarial Class Prompting
• Dual-Thresholding Heatmaps to Cluster Proposals for Weakly Supervised Object Detection
• Mitigating Spurious Correlations in Weakly Supervised Semantic Segmentation via Cross-architecture Consistency Regularization
• Out-of-distribution data supervision towards biomedical semantic segmentation
• Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation

Related

• MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentation
• Fully Using Classifiers for Weakly Supervised Semantic Segmentation with Modified Cues
• Learning to Detour: Shortcut Mitigating Augmentation for Weakly Supervised Semantic Segmentation
• Fine-grained Background Representation for Weakly Supervised Semantic Segmentation
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features