Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation

2019/06/13 by Yunchun Chen, Chen, Yun-Chun, Yen‐Yu Lin +5 · 2 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1906.05857

openalex publication_date 2019/06/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present an approach for jointly matching and segmenting object instances of the same category within a collection of images. In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks. The key insights of our method are two-fold. First, the estimated dense correspondence fields from semantic matching provide supervision for object co-segmentation by enforcing consistency between the predicted masks from a pair of images. Second, the predicted object masks from object co-segmentation in turn allow us to reduce the adverse effects due to background clutters for improving semantic matching. Our model is end-to-end trainable and does not require supervision from manually annotated correspondences and object masks. We validate the efficacy of our approach on five benchmark datasets: TSS, Internet, PF-PASCAL, PF-WILLOW, and SPair-71k, and show that our algorithm performs favorably against the state-of-the-art methods on both semantic matching and object co-segmentation tasks.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• SIFT Flow: Dense Correspondence across Scenes and Its Applications
• A Threshold Selection Method from Gray-Level Histograms
• ImageNet classification with deep convolutional neural networks
• Recent Advance in Content-based Image Retrieval: A Literature Survey
• End-to-End Learning of Geometry and Context for Deep Stereo Regression
• SPair-71k: A Large-scale Benchmark for Semantic Correspondence

Cited by

• Dense Semantic Matching with VGGT Prior
• Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels

Related

• Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation
• Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation
• COMNet: Co-Occurrent Matching for Weakly Supervised Semantic Segmentation
• Cross Language Image Matching for Weakly Supervised Semantic Segmentation
• Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features