Bias Beyond English: Evaluating Social Bias and Debiasing Methods in a Low-Resource Setting

2025/04/15 by Zhou, Ej, Lu, Weiming
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2504.11183

Abstract

Social bias in language models can potentially exacerbate social inequalities. Despite it having garnered wide attention, most research focuses on English data. In a low-resource scenario, the models often perform worse due to insufficient training data. This study aims to leverage high-resource language corpora to evaluate bias and experiment with debiasing methods in low-resource languages. We evaluated the performance of recent multilingual models in five languages: English, Chinese, Russian, Indonesian and Thai, and analyzed four bias dimensions: gender, religion, nationality, and race-color. By constructing multilingual bias evaluation datasets, this study allows fair comparisons between models across languages. We have further investigated three debiasing methods-CDA, Dropout, SenDeb-and demonstrated that debiasing methods from high-resource languages can be effectively transferred to low-resource ones, providing actionable insights for fairness research in multilingual NLP.

Related

• Social Bias in Multilingual Language Models: A Survey
• Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages
• On Evaluating and Mitigating Gender Biases in Multilingual Settings
• Global Voices, Local Biases: Socio-Cultural Prejudices across Languages
• Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques