Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks

2017/11/23 by Nicolas Audebert, Audebert, Nicolas, Bertrand Le Saux +3 · 4 citations
Environmental Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1711.08681

Abstract

In this work, we investigate various methods to deal with semantic labeling of very high resolution multi-modal remote sensing data. Especially, we study how deep fully convolutional networks can be adapted to deal with multi-modal and multi-scale remote sensing data for semantic labeling. Our contributions are threefold: a) we present an efficient multi-scale approach to leverage both a large spatial context and the high resolution data, b) we investigate early and late fusion of Lidar and multispectral data, c) we validate our methods on two public datasets with state-of-the-art results. Our results indicate that late fusion make it possible to recover errors steaming from ambiguous data, while early fusion allows for better joint-feature learning but at the cost of higher sensitivity to missing data.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification

Cited by

• A Spatially Masked Adaptive Gated Network for multimodal post-flood water extent mapping using SAR and incomplete multispectral data
• Vehicle tracks in the UK uplands vary in occurrence with land use and overlap with peatlands and protected areas
• Geospatial Machine Learning Libraries
• AMMNet: An Asymmetric Multi-Modal Network for Remote Sensing Semantic Segmentation

Related

• Dense Dilated Convolutions Merging Network for Semantic Mapping of Remote Sensing Images
• Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
• Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
• More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification
• Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks