Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications

2025/06/24 by Lujun Li, Li, Lujun, Yiqun Wang +3 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Currency Recognition and Detection #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2506.19591

openalex publication_date 2025/06/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Cloud cover in multispectral imagery (MSI) poses significant challenges for early season crop mapping, as it leads to missing or corrupted spectral information. Synthetic aperture radar (SAR) data, which is not affected by cloud interference, offers a complementary solution, but lack sufficient spectral detail for precise crop mapping. To address this, we propose a novel framework, Time-series MSI Image Reconstruction using Vision Transformer (ViT), to reconstruct MSI data in cloud-covered regions by leveraging the temporal coherence of MSI and the complementary information from SAR from the attention mechanism. Comprehensive experiments, using rigorous reconstruction evaluation metrics, demonstrate that Time-series ViT framework significantly outperforms baselines that use non-time-series MSI and SAR or time-series MSI without SAR, effectively enhancing MSI image reconstruction in cloud-covered regions.

Citations

• Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
• ViTs for SITS: Vision Transformers for Satellite Image Time Series
• A machine learning approach for accurate crop type mapping using combined SAR and optical time series data
• Is Space-Time Attention All You Need for Video Understanding?
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A commentary review on the use of normalized difference vegetation index (NDVI) in the era of popular remote sensing
• Attention Is All You Need
• Image quality assessment: from error visibility to structural similarity

Cited by

• Temporal-Spatial Tubelet Embedding for Cloud-Robust MSI Reconstruction using MSI-SAR Fusion: A Multi-Head Self-Attention Video Vision Transformer Approach

Related

• Temporal-Spatial Tubelet Embedding for Cloud-Robust MSI Reconstruction using MSI-SAR Fusion: A Multi-Head Self-Attention Video Vision Transformer Approach
• Seeing Through Clouds in Satellite Images
• Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series
• Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
• Multi-Stage Feature Fusion of Multispectral and SAR Satellite Images for Seasonal Crop-Type Mapping at Regional Scale Using an Adapted 3D U-Net Model