Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation

2016/03/15 by Zhicheng Yan, Hao Zhang, Yan, Zhicheng +7 · 4 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning and Data Classification #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1603.04871

openalex publication_date 2016/03/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

State-of-the-art results of semantic segmentation are established by Fully Convolutional neural Networks (FCNs). FCNs rely on cascaded convolutional and pooling layers to gradually enlarge the receptive fields of neurons, resulting in an indirect way of modeling the distant contextual dependence. In this work, we advocate the use of spatially recurrent layers (i.e. ReNet layers) which directly capture global contexts and lead to improved feature representations. We demonstrate the effectiveness of ReNet layers by building a Naive deep ReNet (N-ReNet), which achieves competitive performance on Stanford Background dataset. Furthermore, we integrate ReNet layers with FCNs, and develop a novel Hybrid deep ReNet (H-ReNet). It enjoys a few remarkable properties, including full-image receptive fields, end-to-end training, and efficient network execution. On the PASCAL VOC 2012 benchmark, the H-ReNet improves the results of state-of-the-art approaches Piecewise, CRFasRNN and DeepParsing by 3.6%, 2.3% and 0.2%, respectively, and achieves the highest IoUs for 13 out of the 20 object classes.

Citations

• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Recurrent Neural Network Regularization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• The Pascal Visual Object Classes (VOC) Challenge
• Fully Connected Deep Structured Networks
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Inside-Outside Net: Detecting Objects in Context with Skip Pooling and\n Recurrent Neural Networks
• Learning Deconvolution Network for Semantic Segmentation

Cited by

• Semantic Correlation Promoted Shape-Variant Context for Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks

Related

• ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
• Efficient piecewise training of deep structured models for semantic segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Improving Fully Convolution Network for Semantic Segmentation