Focusing on Shadows for Predicting Heightmaps from Single Remotely Sensed RGB Images with Deep Learning

2021/04/22 by Savvas Karatsiolis, Karatsiolis, Savvas, Andreas Kamilaris +1
Computer Science · Environmental Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Remote Sensing and LiDAR Applications #Remote Sensing in Agriculture #Video Surveillance and Tracking Methods #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2104.10874

30 pages, 13 figures

arxiv created 2021/04/22 · openalex publication_date 2021/04/22 · arxiv updated 2021/04/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Estimating the heightmaps of buildings and vegetation in single remotely sensed images is a challenging problem. Effective solutions to this problem can comprise the stepping stone for solving complex and demanding problems that require 3D information of aerial imagery in the remote sensing discipline, which might be expensive or not feasible to require. We propose a task-focused Deep Learning (DL) model that takes advantage of the shadow map of a remotely sensed image to calculate its heightmap. The shadow is computed efficiently and does not add significant computation complexity. The model is trained with aerial images and their Lidar measurements, achieving superior performance on the task. We validate the model with a dataset covering a large area of Manchester, UK, as well as the 2018 IEEE GRSS Data Fusion Contest Lidar dataset. Our work suggests that the proposed DL architecture and the technique of injecting shadows information into the model are valuable for improving the heightmap estimation task for single remotely sensed imagery.

Citations

• ImageNet classification with deep convolutional neural networks
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Densely Connected Convolutional Networks
• A Review on Deep Learning Techniques Applied to Semantic Segmentation
• Rethinking Atrous Convolution for Semantic Image Segmentation
• IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network

Related

• Precision in Building Extraction: Comparing Shallow and Deep Models using LiDAR Data
• A Nearest Neighbor Network to Extract Digital Terrain Models from 3D Point Clouds
• IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network
• sat2pc: Estimating Point Cloud of Building Roofs from 2D Satellite Images
• Aerial Height Prediction and Refinement Neural Networks with Semantic and Geometric Guidance