Language (Technology) is Power: A Critical Survey of "Bias" in NLP
2020/05/28 by Su Lin Blodgett, Solon Barocas, Blodgett, Su Lin +5 · 2 voices · 97 citations
#cs.CL #cs.CY
paper · pdf · doi:10.48550/arxiv.2005.14050
Abstract
We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.
Cited by
- Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance
- Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
- Explaining GAND: A Resource on Gender-Ambiguous Natural Data & Contrastive Attribution
- On The Conceptualization and Societal Impact of Cross-Cultural Bias
- Measuring Mechanistic Independence: Can Bias Be Removed Without Erasing Demographics?
- Identifying Features Associated with Bias Against 93 Stigmatized Groups in Language Models and Guardrail Model Safety Mitigation
- Teaching and Critiquing Conceptualization and Operationalization in NLP
- Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation
- Identifying Bias in Machine-generated Text Detection
- What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models
- LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
- A Unifying Human-Centered AI Fairness Framework
- AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models
- BERnaT: Basque Encoders for Representing Natural Textual Diversity
- Understanding Down Syndrome Stereotypes in LLM-Based Personas
- Are LLMs Good Safety Agents or a Propaganda Engine?
- Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses
- Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
- Empathetic Cascading Networks: A Multi-Stage Prompting Technique for Reducing Social Biases in Large Language Models
- Bias in, Bias out: Annotation Bias in Multilingual Large Language Models
- ‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
- Benchmarking Educational LLMs with Analytics: A Case Study on Gender Bias in Feedback
- Annotating Dimensions of Social Perception in Text: A Sentence-Level Dataset of Warmth and Competence
- From Double to Triple Burden: Gender Stratification in the Latin American Data Annotation Gig Economy
- Large Language Models Develop Novel Social Biases Through Adaptive Exploration
- Multi-Reward GRPO Fine-Tuning for De-biasing Large Language Models: A Study Based on Chinese-Context Discrimination Data
- LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis
- Advancing Equitable AI: Evaluating Cultural Expressiveness in LLMs for Latin American Contexts
- Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
- Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens
- Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMs
- Back to the Communities: A Mixed-Methods and Community-Driven Evaluation of Cultural Sensitivity in Text-to-Image Models
- Exploring the Intersection of AI, Language, and Law: A Bibliometric Analysis
- Biases in the Blind Spot: Detecting What LLMs Fail to Mention
- Political censorship in large language models originating from China
- Ideology-Based LLMs for Content Moderation
- Can ChatGPT Code Communication Data Fairly?: Empirical Evidence from Multiple Collaborative Tasks
- "You Are Rejected!": An Empirical Study of Large Language Models Taking Hiring Evaluations
- Identity-Aware Large Language Models require Cultural Reasoning
- Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
- ABLEIST: Intersectional Disability Bias in LLM-Generated Hiring Scenarios
- MarxistLLM: Fine-tuning a language model with a Marxist worldview
- Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions
- Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains
- Large Language Models: A Paradigm Shift for Dementia Diagnosis and Care
- Reward Model Perspectives: Whose Opinions Do Reward Models Reward?
- The fragility of "cultural tendencies" in LLMs
- Assessing Human Rights Risks in AI: A Framework for Model Evaluation
- Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
- Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study
- Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions
- BTC-SAM: Leveraging LLMs for Generation of Bias Test Cases for Sentiment Analysis Models
- Seeing Symbols, Missing Cultures: Probing Vision-Language Models' Reasoning on Fire Imagery and Cultural Meaning
- Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
- Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
- Aligning Recommendations with User Popularity Preferences
- The Sound of Silencing: Identities and Ideologies in Commercial Text-To-Speech
- DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture
- HICode: Hierarchical Inductive Coding with LLMs
- Intrinsic Meets Extrinsic Fairness: Assessing the Downstream Impact of Bias Mitigation in Large Language Models
- Philosophy-informed Machine Learning
- Bias Amplification in Stable Diffusion's Representation of Stigma Through Skin Tones and Their Homogeneity
- A Framework for Generating Artificial Datasets to Validate Absolute and Relative Position Concepts
- Measuring Gender Bias in Job Title Matching for Grammatical Gender Languages
- Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
- PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
- Toward Socially Aware Vision-Language Models: Evaluating Cultural Competence Through Multimodal Story Generation
- From Detection to Mitigation: Addressing Gender Bias in Chinese Texts via Efficient Tuning and Voting-Based Rebalancing
- Augmenting Human-Centered Racial Covenant Detection and Georeferencing with Plug-and-Play NLP Pipelines
- Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
- SESGO: Spanish Evaluation of Stereotypical Generative Outputs
- Applications and Challenges of Fairness APIs in Machine Learning Software
- Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations
- Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Models
- Where Should I Study? Biased Language Models Decide! Evaluating Fairness in LMs for Academic Recommendations
- Culture is Everywhere: A Call for Intentionally Cultural Evaluation
- Social Bias in Multilingual Language Models: A Survey
- Bias-Adjusted LLM Agents for Human-Like Decision-Making via Behavioral Economics
- ISCA: A Framework for Interview-Style Conversational Agents
- Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs
- DAIQ: Auditing Demographic Attribute Inference from Question in LLMs
- Speciesism in AI: Evaluating Discrimination Against Animals in Large Language Models
- Bias is a Math Problem, AI Bias is a Technical Problem: 10-year Literature Review of AI/LLM Bias Research Reveals Narrow [Gender-Centric] Conceptions of 'Bias', and Academia-Industry Gap
- BIPOLAR: Polarization-based granular framework for LLM bias evaluation
- Yet another algorithmic bias: A Discursive Analysis of Large Language Models Reinforcing Dominant Discourses on Gender and Race
- PakBBQ: A Culturally Adapted Bias Benchmark for QA
- "Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas
- Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution
- Fair Play in the Newsroom: Actor-Based Filtering Gender Discrimination in Text Corpora
- I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations
- Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach
- Moving beyond harm. A critical review of how NLP research approaches discrimination
- A Survey on Data Security in Large Language Models
- Bias Association Discovery Framework for Open-Ended LLM Generations
- Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English
- Factual Inconsistencies in Multilingual Wikipedia Tables
- Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making
Discussions
Related