Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning

2025/09/27 by Hui Yu, Yu, Haorui, Zhao Yang +4
Psychology · #Cultural analysis #Cultural background #Cultural diversity #Language, Metaphor, and Cognition #Matching (statistics) #Meaning (existential)

paper · pdf · doi:10.48550/arxiv.2509.23311

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/09/27 · openalex created_date 2025/10/19 · openalex updated_date 2026/08/05

Abstract

Vision-Language Models (VLMs) often appear culturally competent but rely on superficial pattern matching rather than genuine cultural understanding. We introduce a diagnostic framework to probe VLM reasoning on fire-themed cultural imagery through both classification and explanation analysis. Testing multiple models on Western festivals, non-Western traditions, and emergency scenes reveals systematic biases: models correctly identify prominent Western festivals but struggle with underrepresented cultural events, frequently offering vague labels or dangerously misclassifying emergencies as celebrations. These failures expose the risks of symbolic shortcuts and highlight the need for cultural evaluation beyond accuracy metrics to ensure interpretable and fair multimodal systems.

Citations

• An Evaluation of Cultural Value Alignment in LLM
• CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
• Benchmarking Vision Language Models for Cultural Understanding
• Language (Technology) is Power: A Critical Survey of "Bias" in NLP
• A Survey on Bias and Fairness in Machine Learning
• See It from My Perspective: How Language Affects Cultural Bias in Image Understanding

Related

• Vision Language Models are Confused Tourists
• GIMMICK -- Globally Inclusive Multimodal Multitask Cultural Knowledge Benchmarking
• Evaluation of Cultural Competence of Vision-Language Models
• Benchmarking Vision Language Models for Cultural Understanding
• Seeing Culture: A Benchmark for Visual Reasoning and Grounding