Inherent Trade-Offs in the Fair Determination of Risk Scores

2017/01/01 by Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan · 1 voice · 30 citations
Arts and Humanities · Computer Science · #Bayesian Modeling and Causal Inference #Explainable Artificial Intelligence (XAI) #Philosophy and History of Science

paper · pdf · doi:10.4230/lipics.itcs.2017.43

openalex publication_date 2017/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.

Cited by

• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
• A Reductions Approach to Fair Classification
• A Multi-Objective Evaluation Framework for Analyzing Utility-Fairness Trade-Offs in Machine Learning Systems
• Feedback Effects in Repeat-Use Criminal Risk Assessments
• "I think this is fair'': Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment
• The Statistical Fairness-Accuracy Frontier
• Breaking the Cycle of Incarceration With Targeted Mental Health Outreach: A Case Study in Machine Learning for Public Policy
• Ethical and social considerations of applying artificial intelligence in healthcare—a two-pronged scoping review
• Comparing Fairness Criteria Based on Social Outcome
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and\n Mitigating Unwanted Algorithmic Bias
• Aligning AI With Shared Human Values
• Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
• Top-label calibration and multiclass-to-binary reductions
• An Information-Theoretic Framework for Credit Risk Modeling: Unifying Industry Practice with Statistical Theory for Fair and Interpretable Scorecards
• Evaluating Fairness Metrics in the Presence of Dataset Bias
• Counterfactual Reasoning for Fair Clinical Risk Prediction
• Sequential Cohort Selection
• Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints
• Towards Out-Of-Distribution Generalization: A Survey
• Towards Fairness Certification in Artificial Intelligence
• Counterfactual Fairness
• Bridging Research Gaps Between Academic Research and Legal Investigations of Algorithmic Discrimination
• Fairness in rankings and recommendations: an overview
• A Convex Framework for Fair Regression
• Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attack
• Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI
• Decomposing Global AUC into Cluster-Level Contributions for Localized Model Diagnostics
• The Fair Game: Auditing & Debiasing AI Algorithms Over Time
• FairPOT: Balancing AUC Performance and Fairness with Proportional Optimal Transport
• Are All Genders Equal in the Eyes of Algorithms? -- Analysing Search and Retrieval Algorithms for Algorithmic Gender Fairness

Discussions

• dx.doi.org/10.4230/LIPI... doi.org/10.1093/llc/... [bsky, 1 points, 0 comments]

Related

• Inherent Trade-Offs in the Fair Determination of Risk Scores
• The cost of fairness in classification
• Fairness criteria through the lens of directed acyclic graphical models
• Accuracy vs. Accuracy: Computational Tradeoffs Between Classification Rates and Utility
• Obtaining fairness using optimal transport theory