MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation

2021/03/24 by Guangwei Gao, Gao, Guangwei, Guoan Xu +9 · 9 citations
Computer Science · #Advanced Neural Network Applications #Algorithm #Artificial intelligence #Artificial neural network #Computer Vision and Pattern Recognition (cs.CV) #Computer engineering #Computer science #Computer vision #Context (archaeology) #Convolution (computer science) #Domain Adaptation and Few-Shot Learning #Encoder #FOS: Computer and information sciences #Inference #Real-time computing #Residual #Segmentation #Video Surveillance and Tracking Methods #cs.CV

paper · pdf · doi:10.48550/arxiv.2103.13044

published in arXiv (Cornell University) (Cornell University) · IEEE Transactions on Intelligent Transportation Systems, 11 pages, 7 figures

openalex publication_date 2021/03/24 · openalex created_date 2021/03/29 · arxiv created 2021/07/16 · arxiv updated 2021/07/19 · openalex updated_date 2026/07/28

Abstract

In recent years, how to strike a good trade-off between accuracy and inference speed has become the core issue for real-time semantic segmentation applications, which plays a vital role in real-world scenarios such as autonomous driving systems and drones. In this study, we devise a novel lightweight network using a multi-scale context fusion (MSCFNet) scheme, which explores an asymmetric encoder-decoder architecture to dispose this problem. More specifically, the encoder adopts some developed efficient asymmetric residual (EAR) modules, which are composed of factorization depth-wise convolution and dilation convolution. Meanwhile, instead of complicated computation, simple deconvolution is applied in the decoder to further reduce the amount of parameters while still maintaining high segmentation accuracy. Also, MSCFNet has branches with efficient attention modules from different stages of the network to well capture multi-scale contextual information. Then we combine them before the final classification to enhance the expression of the features and improve the segmentation efficiency. Comprehensive experiments on challenging datasets have demonstrated that the proposed MSCFNet, which contains only 1.15M parameters, achieves 71.9% Mean IoU on the Cityscapes testing dataset and can run at over 50 FPS on a single Titan XP GPU configuration.

Citations

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Fast-SCNN: Fast Semantic Segmentation Network
• High-Resolution Representations for Labeling Pixels and Regions
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
• ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

Cited by

• Prototypical Progressive Alignment and Reweighting for Generalizable Semantic Segmentation
• ECMNet:Lightweight Semantic Segmentation with Efficient CNN-Mamba Network
• Building Lightweight Semantic Segmentation Models for Aerial Images Using Dual Relation Distillation

Related

• Feature Reuse and Fusion for Real-time Semantic segmentation
• FarSee-Net: Real-Time Semantic Segmentation by Efficient Multi-scale Context Aggregation and Feature Space Super-resolution
• Feature Pyramid Encoding Network for Real-time Semantic Segmentation
• ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time
• ESNet: An Efficient Symmetric Network for Real-time Semantic Segmentation