SPGNet: Semantic Prediction Guidance for Scene Parsing

2019/08/26 by Bowen Cheng, Cheng, Bowen, Liang-Chieh Chen +14 · 2 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Video Surveillance and Tracking Methods

paper · pdf · doi:10.48550/arxiv.1908.09798

openalex publication_date 2019/08/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multi-scale context module and single-stage encoder-decoder structure are commonly employed for semantic segmentation. The multi-scale context module refers to the operations to aggregate feature responses from a large spatial extent, while the single-stage encoder-decoder structure encodes the high-level semantic information in the encoder path and recovers the boundary information in the decoder path. In contrast, multi-stage encoder-decoder networks have been widely used in human pose estimation and show superior performance than their single-stage counterpart. However, few efforts have been attempted to bring this effective design to semantic segmentation. In this work, we propose a Semantic Prediction Guidance (SPG) module which learns to re-weight the local features through the guidance from pixel-wise semantic prediction. We find that by carefully re-weighting features across stages, a two-stage encoder-decoder network coupled with our proposed SPG module can significantly outperform its one-stage counterpart with similar parameters and computations. Finally, we report experimental results on the semantic segmentation benchmark Cityscapes, in which our SPGNet attains 81.1% on the test set using only 'fine' annotations.

Citations

• The Pascal Visual Object Classes (VOC) Challenge
• Backpropagation Applied to Handwritten Zip Code Recognition
• ImageNet classification with deep convolutional neural networks
• Bridging Category-level and Instance-level Semantic Image Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Associative Embedding: End-to-End Learning for Joint Detection and\n Grouping
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Stacked U-Nets: A No-Frills Approach to Natural Image Segmentation
• SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Rethinking on Multi-Stage Networks for Human Pose Estimation
• Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
• DeeperLab: Single-Shot Image Parser
• Fully Convolutional Networks for Semantic Segmentation

Cited by

• Rethinking Query-based Transformer for Continual Image Segmentation
• The Missing Point in Vision Transformers for Universal Image Segmentation

Related

• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
• SegMAN: Omni-scale Context Modeling with State Space Models and Local Attention for Semantic Segmentation
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor
• Simple and Efficient Architectures for Semantic Segmentation