Nuanced Metrics for Measuring Unintended Bias with Real Data for Text\n Classification

2019/03/11 by Daniel Borkan, Borkan, Daniel, Lucas Dixon +7 · 30 citations
Social Sciences · Computer Science · #Misinformation and Its Impacts #Hate Speech and Cyberbullying Detection

paper · pdf · doi:10.48550/arxiv.1903.04561

Abstract

Unintended bias in Machine Learning can manifest as systemic differences in\nperformance for different demographic groups, potentially compounding existing\nchallenges to fairness in society at large. In this paper, we introduce a suite\nof threshold-agnostic metrics that provide a nuanced view of this unintended\nbias, by considering the various ways that a classifier's score distribution\ncan vary across designated groups. We also introduce a large new test set of\nonline comments with crowd-sourced annotations for identity references. We use\nthis to show how our metrics can be used to find new and potentially subtle\nunintended bias in existing public models.\n

Cited by

• Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples
• Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
• The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluation
• Measuring Uncertainty Calibration
• Representation Invariance and Allocation: When Subgroup Balance Matters
• Provably Safe Model Updates
• On the Unreasonable Effectiveness of Last-layer Retraining
• Prompt Fairness: Sub-group Disparities in LLMs
• Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection
• Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Explaining the Deep Natural Language Processing by Mining Textual\n Interpretable Features
• Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
• Charting the European LLM Benchmarking Landscape: A New Taxonomy and a Set of Best Practices
• Knowledge-guided Continual Learning for Behavioral Analytics Systems
• Doubly-Regressing Approach for Subgroup Fairness
• Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detection
• Test-Time Adaptation by Causal Trimming
• Myopic Bayesian Decision Theory for Batch Active Learning with Partial Batch Label Sampling
• Promptimizer: User-Led Prompt Optimization for Personal Content Classification
• Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
• Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
• Longitudinal Monitoring of LLM Content Moderation of Social Issues
• LLM Chatbot-Creation Approaches
• Fairness for the People, by the People: Minority Collective Action
• Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing
• GANDiff FR: Hybrid GAN Diffusion Synthesis for Causal Bias Attribution in Face Recognition
• Simple data balancing achieves competitive worst-group-accuracy
• I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations
• FairPOT: Balancing AUC Performance and Fairness with Proportional Optimal Transport
• Consensus-Driven Active Model Selection

Related

• Measuring Fairness of Text Classifiers via Prediction Sensitivity
• BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
• Intrinsic Bias Metrics Do Not Correlate with Application Bias
• Uncovering Latent Biases in Text: Method and Application to Peer Review
• Algorithmic Accountability in Small Data: Sample-Size-Induced Bias Within Classification Metrics