From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test

2025/05/24 by Zhou Li, Dai, Xunlian, Zhou, Li +3 · 1 citation
Health Professions · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Interpreting and Communication in Healthcare #Multilingual Education and Policy

paper · pdf · doi:10.48550/arxiv.2505.18562

openalex publication_date 2025/05/24 · openalex created_date 2025/09/28 · openalex updated_date 2026/07/28

Abstract

The human-centered word association test (WAT) serves as a cognitive proxy, revealing sociocultural variations through culturally shared semantic expectations and implicit linguistic patterns shaped by lived experiences. We extend this test into an LLM-adaptive, free-relation task to assess the alignment of large language models (LLMs) with cross-cultural cognition. To address culture preference, we propose CultureSteer, an innovative approach that moves beyond superficial cultural prompting by embedding cultural-specific semantic associations directly within the model's internal representation space. Experiments show that current LLMs exhibit significant bias toward Western (notably American) schemas at the word association level. In contrast, our model substantially improves cross-cultural alignment, capturing diverse semantic associations. Further validation on culture-sensitive downstream tasks confirms its efficacy in fostering cognitive alignment across cultures. This work contributes a novel methodological paradigm for enhancing cultural awareness in LLMs, advancing the development of more inclusive language technologies.

Citations

• CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimization
• Effectively Steer LLM To Follow Preference via Building Confident Directions
• Culture is Not Trivia: Sociocultural Theory for Cultural NLP
• Specializing Large Language Models to Simulate Survey Response Distributions for Global Populations
• Self-Pluralising Culture Alignment for Large Language Models
• Can Language Models Reason about Individualistic Human Values and Preferences?
• Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
• The Llama 3 Herd of Models
• Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models?
• External Invariants: A Cryptographic Trust Architecture for Institutional AI Inference
• BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
• Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization
• Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense
• CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies
• MuLan: A Study of Fact Mutability in Language Models
• Investigating Cultural Alignment of Large Language Models
• Assessing LLMs for Moral Value Pluralism
• Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs
• Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions
• Towards Measuring the Representation of Subjective Global Opinions in Language Models
• Word Embeddings Are Steers for Language Models
• Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study
• Democratizing Neural Machine Translation with OPUS-MT
• Persistent Anti-Muslim Bias in Large Language Models
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Steerable Chatbots: Exploring Personalization Control Interfaces via LLM Activation Steering

Cited by

• ALIGN: Word Association Learning for Cultural Alignment in Large Language Models

Related

• ALIGN: Word Association Learning for Cultural Alignment in Large Language Models
• Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
• Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions
• How Well Do LLMs Identify Cultural Unity in Diversity?
• Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models