vix.ing · top · new · best · stats · spec

Exploring Spatial Schema Intuitions in Large Language and Vision Models

2024/02/01 by Philipp Wicke, Wicke, Philipp, Lennart Wachowiak +1 · 1 citation
Computer Science · Social Sciences · #Computation and Language (cs.CL) #Constraint Satisfaction and Optimization #FOS: Computer and information sciences #Geographic Information Systems Studies #Speech and dialogue systems

paper · pdf · doi:10.48550/arxiv.2402.00956

openalex publication_date 2024/02/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Despite the ubiquity of large language models (LLMs) in AI research, the question of embodiment in LLMs remains underexplored, distinguishing them from embodied systems in robotics where sensory perception directly informs physical action. Our investigation navigates the intriguing terrain of whether LLMs, despite their non-embodied nature, effectively capture implicit human intuitions about fundamental, spatial building blocks of language. We employ insights from spatial cognitive foundations developed through early sensorimotor experiences, guiding our exploration through the reproduction of three psycholinguistic experiments. Surprisingly, correlations between model outputs and human responses emerge, revealing adaptability without a tangible connection to embodied experiences. Notable distinctions include polarized language model responses and reduced correlations in vision language models. This research contributes to a nuanced understanding of the interplay between language, spatial experiences, and the computations made by large language models. More at https://cisnlp.github.io/SpatialSchemas/

Cited by

Related