Fairness in Large Language Models: A Taxonomic Survey

2024/03/31 by Zhibo Chu, Chu, Zhibo, Zichong Wang +3 · 19 citations
Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Ethics and Social Impacts of AI #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2404.01349

openalex publication_date 2024/03/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have demonstrated remarkable success across various domains. However, despite their promising performance in numerous real-world applications, most of these algorithms lack fairness considerations. Consequently, they may lead to discriminatory outcomes against certain communities, particularly marginalized populations, prompting extensive study in fair LLMs. On the other hand, fairness in LLMs, in contrast to fairness in traditional machine learning, entails exclusive backgrounds, taxonomies, and fulfillment techniques. To this end, this survey presents a comprehensive overview of recent advances in the existing literature concerning fair LLMs. Specifically, a brief introduction to LLMs is provided, followed by an analysis of factors contributing to bias in LLMs. Additionally, the concept of fairness in LLMs is discussed categorically, summarizing metrics for evaluating bias in LLMs and existing algorithms for promoting fairness. Furthermore, resources for evaluating bias in LLMs, including toolkits and datasets, are summarized. Finally, existing research challenges and open questions are discussed.

Cited by

• Responsible Intelligence in Practice: A Fairness Audit of Open Large Language Models for Library Reference Services
• Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation
• Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relations
• Fairness-Aware Graph Representation Learning with Limited Demographic Information
• Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs
• The Disparate Impacts of Speculative Decoding
• CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMs
• Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformers
• Towards Transparent AI: A Survey on Explainable Language Models
• Controllable Pareto Trade-off between Fairness and Accuracy
• AMCR: A Framework for Assessing and Mitigating Copyright Risks in Generative Models
• Integrating gender inclusivity into large language models via instruction tuning
• Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs
• BIPOLAR: Polarization-based granular framework for LLM bias evaluation
• A Close Reading Approach to Gender Narrative Biases in AI-Generated Stories
• FairFLRep: Fairness aware fault localization and repair of Deep Neural Networks
• Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models

Related

• Fairness in Large Language Models in Three Hours
• A Survey on Fairness in Large Language Models
• Bias and Fairness in Large Language Models: A Survey
• Bias and Fairness in Large Language Models: A Survey
• Datasets for Fairness in Language Models: An In-Depth Survey