Intersectional Bias in Japanese Large Language Models from a Contextualized Perspective

2025/06/14 by Yanaka, Hitomi, He, Xinqi, Lu, Jie +6
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2506.12327

Abstract

An increasing number of studies have examined the social bias of rapidly developed large language models (LLMs). Although most of these studies have focused on bias occurring in a single social attribute, research in social science has shown that social bias often occurs in the form of intersectionality -- the constitutive and contextualized perspective on bias aroused by social attributes. In this study, we construct the Japanese benchmark inter-JBBQ, designed to evaluate the intersectional bias in LLMs on the question-answering setting. Using inter-JBBQ to analyze GPT-4o and Swallow, we find that biased output varies according to its contexts even with the equal combination of social attributes.

Citations

• Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
• Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?
• Bias and Fairness in Large Language Models: A Survey
• Bias and Fairness in Large Language Models: A Survey
• mdx: A Cloud Platform for Supporting Data Science and Cross-Disciplinary Research Collaborations
• BBQ: A Hand-Built Bias Benchmark for Question Answering
• Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models
• Language (Technology) is Power: A Critical Survey of "Bias" in NLP
• Race as biology is fiction, racism as a social problem is real: Anthropological and historical perspectives on the social construction of race.

Related

• JBBQ: Japanese Bias Benchmark for Analyzing Social Biases in Large Language Models
• Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution
• Intersectional Bias in Causal Language Models
• Algorithmic reproduction of social inequality: language attitude bias in Chinese and English pre-trained language models
• Mapping the Multilingual Margins: Intersectional Biases of Sentiment Analysis Systems in English, Spanish, and Arabic