Weakly Supervised Semantic Segmentation using Web-Crawled Videos

2017/01/02 by Seunghoon Hong, Donghun Yeo, Hong, Seunghoon +7
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.1701.00352

openalex publication_date 2017/01/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

We propose a novel algorithm for weakly supervised semantic segmentation based on image-level class labels only. In weakly supervised setting, it is commonly observed that trained model overly focuses on discriminative parts rather than the entire object area. Our goal is to overcome this limitation with no additional human intervention by retrieving videos relevant to target class labels from web repository, and generating segmentation labels from the retrieved videos to simulate strong supervision for semantic segmentation. During this process, we take advantage of image classification with discriminative localization technique to reject false alarms in retrieved videos and identify relevant spatio-temporal volumes within retrieved videos. Although the entire procedure does not require any additional supervision, the segmentation annotations obtained from videos are sufficiently strong to learn a model for semantic segmentation. The proposed algorithm substantially outperforms existing methods based on the same level of supervision and is even as competitive as the approaches relying on extra annotations.

Citations

• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• The Pascal Visual Object Classes (VOC) Challenge
• Semantic Image Segmentation via Deep Parsing Network
• "GrabCut"
• Overview of the H.264/AVC video coding standard
• Object Detection, Tracking, and Motion Segmentation for Object-level Video Segmentation
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Webly Supervised Learning of Convolutional Networks
• Learning Deconvolution Network for Semantic Segmentation

Related

• Frame-to-Frame Aggregation of Active Regions in Web Videos for Weakly\n Supervised Semantic Segmentation
• Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation
• NoPeopleAllowed: The Three-Step Approach to Weakly Supervised Semantic Segmentation
• Fully Using Classifiers for Weakly Supervised Semantic Segmentation with Modified Cues
• Learning from the Web: Language Drives Weakly-Supervised Incremental Learning for Semantic Segmentation