Quantifying Geospatial in the Common Crawl Corpus

2024/06/07 by Ilya Ilyankou, Ilyankou, Ilya, Meihui Wang +5 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2406.04952

openalex publication_date 2024/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) exhibit emerging geospatial capabilities, stemming from their pre-training on vast unlabelled text datasets that are often derived from the Common Crawl (CC) corpus. However, the geospatial content within CC remains largely unexplored, impacting our understanding of LLMs' spatial reasoning. This paper investigates the prevalence of geospatial data in recent Common Crawl releases using Gemini 1.5, a powerful language model. By analyzing a sample of documents and manually revising the results, we estimate that 18.7% of web documents in CC contain geospatial information such as coordinates and addresses. We find little difference in prevalence between Enlgish- and non-English-language documents. Our findings provide quantitative insights into the nature and extent of geospatial data in CC, and lay the groundwork for future studies of geospatial biases of LLMs.

Cited by

• Coordinates from Context: Using LLMs to Ground Complex Location References
• Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspective

Related

• Are Large Language Models Geospatially Knowledgeable?
• The World As Large Language Models See It: Exploring the reliability of LLMs in representing geographical features
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models
• Geospatial Mechanistic Interpretability of Large Language Models
• A systematic review of geospatial location embedding approaches in large language models: A path to spatial AI systems