2023/04/05 by Burak Ekim, Michael Schmitt, Ekim, Burak +1 · 1 citation
Environmental Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Remote Sensing and LiDAR Applications #Remote Sensing in Agriculture #Species Distribution and Climate Change
paper · pdf · doi:10.48550/arxiv.2304.02407
openalex publication_date 2023/04/05 · openalex created_date 2023/04/07 · openalex updated_date 2026/07/28
Jointly harnessing complementary features of multi-modal input data in a common latent space has been found to be beneficial long ago. However, the influence of each modality on the models decision remains a puzzle. This study proposes a deep learning framework for the modality-level interpretation of multimodal earth observation data in an end-to-end fashion. While leveraging an explainable machine learning method, namely Occlusion Sensitivity, the proposed framework investigates the influence of modalities under an early-fusion scenario in which the modalities are fused before the learning process. We show that the task of wilderness mapping largely benefits from auxiliary data such as land cover and night time light data.