2024/06/23 by Xixian Yong, Xiao Zhou, Yong, Xixian +1 · 4 citations
Engineering · Social Sciences · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Human Mobility and Location-Based Analysis #Machine Learning (cs.LG) #Traffic Prediction and Management Techniques
paper · pdf · doi:10.48550/arxiv.2407.09523
openalex publication_date 2024/06/23 · openalex created_date 2024/07/17 · openalex updated_date 2026/07/28
Predicting socioeconomic indicators within urban regions is crucial for fostering inclusivity, resilience, and sustainability in cities and human settlements. While pioneering studies have attempted to leverage multi-modal data for socioeconomic prediction, jointly exploring their underlying semantics remains a significant challenge. To address the gap, this paper introduces a Multi-Semantic Contrastive Learning (MuseCL) framework for fine-grained urban region profiling and socioeconomic prediction. Within this framework, we initiate the process by constructing contrastive sample pairs for street view and remote sensing images, capitalizing on the similarities in human mobility and Point of Interest (POI) distribution to derive semantic features from the visual modality. Additionally, we extract semantic insights from POI texts embedded within these regions, employing a pre-trained text encoder. To merge the acquired visual and textual features, we devise an innovative cross-modality-based attentional fusion module, which leverages a contrastive mechanism for integration. Experimental results across multiple cities and indicators consistently highlight the superiority of MuseCL, demonstrating an average improvement of 10% in R2 compared to various competitive baseline models. The code of this work is publicly available at https://github.com/XixianYong/MuseCL.