Exploiting Spatial-Temporal Semantic Consistency for Video Scene Parsing

2021/09/06 by Xingjian He, He, Xingjian, Ning Wang +9
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Video Analysis and Summarization

paper · pdf · doi:10.48550/arxiv.2109.02281

openalex publication_date 2021/09/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Compared with image scene parsing, video scene parsing introduces temporal information, which can effectively improve the consistency and accuracy of prediction. In this paper, we propose a Spatial-Temporal Semantic Consistency method to capture class-exclusive context information. Specifically, we design a spatial-temporal consistency loss to constrain the semantic consistency in spatial and temporal dimensions. In addition, we adopt an pseudo-labeling strategy to enrich the training dataset. We obtain the scores of 59.84% and 58.85% mIoU on development (test part 1) and testing set of VSPW, respectively. And our method wins the 1st place on VSPW challenge at ICCV2021.

Citations

• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Improved Baselines with Momentum Contrastive Learning
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows

Related

• Semantic Segmentation on VSPW Dataset through Contrastive Loss and Multi-dataset Training Approach
• Memory Based Video Scene Parsing
• Static-Dynamic Class-level Perception Consistency in Video Semantic Segmentation
• A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects
• Video Scene Parsing with Predictive Feature Learning