Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

2019/03/11 by Daniel Borkan, Borkan, Daniel, Lucas Dixon +7 · 62 citations
Computer Science · Mathematics · Social Sciences · #Hate Speech and Cyberbullying Detection #Misinformation and Its Impacts #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1903.04561

Updated to fix typo in Equation 4

arxiv created 2019/05/08 · arxiv updated 2019/05/09

Abstract

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by considering the various ways that a classifier's score distribution can vary across designated groups. We also introduce a large new test set of online comments with crowd-sourced annotations for identity references. We use this to show how our metrics can be used to find new and potentially subtle unintended bias in existing public models.

Cited by

• Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples
• Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
• The Erasure Illusion: Stress-Testing the Generalization of LLM Forgetting Evaluation
• Measuring Uncertainty Calibration
• Representation Invariance and Allocation: When Subgroup Balance Matters
• Provably Safe Model Updates
• On the Unreasonable Effectiveness of Last-layer Retraining
• Prompt Fairness: Sub-group Disparities in LLMs
• Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection
• Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustness
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Explaining the Deep Natural Language Processing by Mining Textual Interpretable Features
• Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations
• Charting the European LLM Benchmarking Landscape: A New Taxonomy and a Set of Best Practices
• Knowledge-guided Continual Learning for Behavioral Analytics Systems
• Doubly-Regressing Approach for Subgroup Fairness
• Algorithmic Fairness in NLP: Persona-Infused LLMs for Human-Centric Hate Speech Detection
• Test-Time Adaptation by Causal Trimming
• Myopic Bayesian Decision Theory for Batch Active Learning with Partial Batch Label Sampling
• Promptimizer: User-Led Prompt Optimization for Personal Content Classification
• Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
• Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
• Longitudinal Monitoring of LLM Content Moderation of Social Issues
• LLM Chatbot-Creation Approaches
• Fairness for the People, by the People: Minority Collective Action
• Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing
• GANDiff FR: Hybrid GAN Diffusion Synthesis for Causal Bias Attribution in Face Recognition
• Simple data balancing achieves competitive worst-group-accuracy
• I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations
• FairPOT: Balancing AUC Performance and Fairness with Proportional Optimal Transport
• Consensus-Driven Active Model Selection
• From Benchmarks to Skills: Low-Rank Factors for LLM Evaluation
• CarD-T: an automated pipeline for the nomination and analysis of potential human carcinogens
• The Impact of Coreset Selection on Spurious Correlations and Group Robustness
• Guiding LLM Decision-Making with Fairness Reward Models
• Mitigating Shortcut Learning with InterpoLated Learning
• Incorporating Interventional Independence Improves Robustness against Interventional Distribution Shift
• Argument-Based Consistency in Toxicity Explanations of LLMs
• Web(er) of Hate: A Survey on How Hate Speech Is Typed
• Statistical Hypothesis Testing for Auditing Robustness in Language Models
• ODD: Overlap-aware Estimation of Model Performance under Distribution Shift
• Improving Group Robustness on Spurious Correlation via Evidential Alignment
• IF-GUIDE: Influence Function-Guided Detoxification of LLMs
• Domain Pre-training Impact on Representations
• Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems
• Probing Association Biases in LLM Moderation Over-Sensitivity
• Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift
• NeuronTune: Towards Self-Guided Spurious Bias Mitigation
• The Rich and the Simple: On the Implicit Bias of Adam and SGD
• Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings
• Subgroups Matter for Robust Bias Mitigation
• Teaching Models to Understand (but not Generate) High-risk Data
• Discretization-free Multicalibration through Loss Minimization over Tree Ensembles
• Causal-Invariant Cross-Domain Out-of-Distribution Recommendation
• Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value
• ShortcutProbe: Probing Prediction Shortcuts for Learning Robust Models
• Enforcing Fair Predicted Scores on Intervals of Percentiles by Difference-of-Convex Constraints
• Fine-Grained Class-Conditional Distribution Balancing for Debiased Learning
• On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
• LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation
• Trust The Typical
• CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits

Related

• Measuring Fairness of Text Classifiers via Prediction Sensitivity
• BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
• Intrinsic Bias Metrics Do Not Correlate with Application Bias
• Uncovering Latent Biases in Text: Method and Application to Peer Review
• Algorithmic Accountability in Small Data: Sample-Size-Induced Bias Within Classification Metrics