Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models

2025/06/12 by Aleksandra Sorokovikova, Pavel Chizhov, Sorokovikova, Aleksandra +5 · 5 voices · 5 citations
Computer Science · Medicine · Social Sciences · #Artificial Intelligence in Healthcare and Education #Ethics and Social Impacts of AI #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2506.10491

openalex publication_date 2025/06/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Modern language models are trained on large amounts of data. These data inevitably include controversial and stereotypical content, which contains all sorts of biases related to gender, origin, age, etc. As a result, the models express biased points of view or produce different results based on the assigned personality or the personality of the user. In this paper, we investigate various proxy measures of bias in large language models (LLMs). We find that evaluating models with pre-prompted personae on a multi-subject benchmark (MMLU) leads to negligible and mostly random differences in scores. However, if we reformulate the task and ask a model to grade the user's answer, this shows more significant signs of bias. Finally, if we ask the model for salary negotiation advice, we see pronounced bias in the answers. With the recent trend for LLM assistant memory and personalization, these problems open up from a different angle: modern LLM users do not need to pre-prompt the description of their persona since the model already knows their socio-demographics.

Citations

• Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages

Cited by

• Gender Bias in Emotion Recognition by Large Language Models
• Thought Branches: Interpreting LLM Reasoning Requires Resampling
• A word association network methodology for evaluating implicit biases in LLMs compared to humans
• Race and Gender in LLM-Generated Personas: A Large-Scale Audit of 41 Occupations
• Do Bias Benchmarks Generalise? Evidence from Voice-based Evaluation of Gender Bias in SpeechLLMs

Discussions

• Un estudio realizado en Alemania muestra que los chatbots de IA generativa aconsejaban a las mujeres solicitar salarios significativamente inferiores a los de los hombres con perfiles idénticos arxiv. [bsky, 38 points, 2 comments]
• Lmao, absolutely not surprised. What's interesting in the actual paper in regards to junior roles. The models seem to suggest similar wages. But when negotiating for a senior wage. The difference is e [bsky, 2 points, 0 comments]
• @wolvendamien.bsky.social New study for ya? arxiv.org/pdf/2506.10491 [bsky, 2 points, 1 comments]
• #EconSky Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models https://arxiv.org/pdf/2506.10491 "… the estimation of socio-economic parameters shows substantially more bias than [bsky, 1 points, 2 comments]
• #ContraAI 🤖 #LLMs are biased towards gender and race. In other words: they duplicate the inequalities that exist in real life. This study shows how a language model’s hidden bias leads to suggesting [bsky, 1 points, 1 comments]

Related

• Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation
• Bias and Fairness in Large Language Models: A Survey
• Bias and Fairness in Large Language Models: A Survey
• No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models
• Fine-Grained Bias Detection in LLM: Enhancing detection mechanisms for nuanced biases