A transformer-based multi-task deep learning model for urban livability evaluation by fusing remote sensing and textual geospatial data

2026/01/08 by Wen Zhou, Claudio Persello, Dongping Ming +2 · 1 voice
Engineering · Social Sciences · #Human Mobility and Location-Based Analysis #Remote-Sensing Image Classification #Smart Cities and Technologies

paper · doi:10.1016/j.rse.2026.115232

openalex publication_date 2026/01/08 · openalex created_date 2026/01/09 · openalex updated_date 2026/07/02

Abstract

Livable cities enhance urban economic development, improve physical and mental health, foster well-being, and foster urban sustainability. Evaluating urban livability is therefore important for policymakers to develop urban planning and development strategies aimed at improving livability. Mainstream methods of evaluating urban livability assign different weights to diverse indicators extracted from survey data, statistical data, and geospatial data. To relieve such time-consuming and labor-intensive data collection, this study proposes a transformer-based multi-task multimodal regression (TMTMR) model for the simultaneous evaluation of urban livability focusing on five domain-specific scores. Pretrained state-of-the-art computer vision and natural language processing models serve as backbones to extract features from high spatial resolution remote sensing (RS) images, digital surface models (DSM), night light remote sensing (NLRS) images and point of interest (POI) data. An attention mechanism helps the TMTMR model to assign varying significance levels to features from different modalities, thus capturing both intrinsic information and interrelationships among modalities for livability evaluation. Focusing on 13 Dutch areas, our research demonstrates that the TMTMR model efficiently evaluates urban livability with correlation coefficients ranging from 0.605 to 0.779, and root mean square error values between 0.070 and 0.112 in four unseen test areas. Furthermore, we analyze the synergy between different modalities. We found that modalities of urban livability can be effectively evaluated by aligning, in a descending order, contributions from RS images, NLRS images, DSM, and POI data. We demonstrated that the proposed TMTMR model is capable of effectively evaluating urban livability directly from multimodal geospatial data.

Citations

• AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data
• TerraMind: Large-Scale Generative Multimodality for Earth Observation
• A Multi-Modal Deep Learning Based Approach for House Price Prediction
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Concept Bottleneck Models
• Attention Is All You Need
• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry\n and Semantics
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
• Densely Connected Convolutional Networks
• Deep Residual Learning for Image Recognition
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Discussions

• 🌟 New publication from the NSF I-GUIDE Team! "A transformer-based multi-task deep learning model for urban livability evaluation by fusing remote sensing and textual geospatial data" Read the full p [bsky, 1 points, 0 comments]

Related

• UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction
• CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensing
• UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective
• CityRiSE: Reasoning Urban Socio-Economic Status in Vision-Language Models via Reinforcement Learning
• StreetviewLLM: Extracting Geographic Information Using a Chain-of-Thought Multimodal Large Language Model