Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI

2020/01/01 by Sandra Wachter, Brent Mittelstadt, Chris Russell · 1 voice · 3 citations
Computer Science · Social Sciences · #Bridging (networking) #Corporate governance #Disadvantaged #Discrimination and Equality Law #Disparate impact #Economic Justice #Ethics and Social Impacts of AI #Harm #Jurisprudence #Law, AI, and Intellectual Property #Normative #Relevance (law) #cs.AI

paper · pdf · doi:10.2139/ssrn.3547922

openalex publication_date 2020/01/01 · openalex created_date 2020/03/23 · arxiv created 2020/05/12 · arxiv published 2020/05/12 · arxiv updated 2020/05/13 · openalex updated_date 2026/08/05

Abstract

This article identifies a critical incompatibility between European notions of discrimination and existing statistical measures of fairness. First, we review the evidential requirements to bring a claim under EU non-discrimination law. Due to the disparate nature of algorithmic and human discrimination, the EU's current requirements are too contextual, reliant on intuition, and open to judicial interpretation to be automated. Second, we show how the legal protection offered by non-discrimination law is challenged when AI, not humans, discriminate. Humans discriminate due to negative attitudes (e.g. stereotypes, prejudice) and unintentional biases (e.g. organisational practices or internalised stereotypes) which can act as a signal to victims that discrimination has occurred. Finally, we examine how existing work on fairness in machine learning lines up with procedures for assessing cases under EU non-discrimination law. We propose "conditional demographic disparity" (CDD) as a standard baseline statistical measurement that aligns with the European Court of Justice's "gold standard." Establishing a standard set of statistical evidence for automated discrimination cases can help ensure consistent procedures for assessment, but not judicial interpretation, of cases involving AI and automated systems. Through this proposal for procedural regularity in the identification and assessment of automated discrimination, we clarify how to build considerations of fairness into automated systems as far as possible while still respecting and enabling the contextual approach to judicial interpretation practiced under EU non-discrimination law. N.B. Abridged abstract

Citations

• Fairness through awareness
• Does mitigating ML's impact disparity require treatment disparity?
• Fairness and Abstraction in Sociotechnical Systems

Cited by

• Human, Algorithm, or Both? Gender Bias in Human-Augmented Recruiting
• Co-designing for Compliance: Multi-party Computation Protocols for Post-Market Fairness Monitoring in Algorithmic Hiring
• Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models

Discussions

• Fairness in AI can't be fully automated: - EU's non-discrimination laws rely on context and judicial interpretation, not easily automated. - Algorithmic bias differs from human bias, lacking clear sig [bsky, 2 points, 0 comments]

Related

• Compatibility of Fairness Metrics with EU Non-Discrimination Laws: Demographic Parity & Conditional Demographic Disparity
• Formalising Anti-Discrimination Law in Automated Decision Systems
• Strengthening legal protection against discrimination by algorithms and artificial intelligence
• Implications of the AI Act for Non-Discrimination Law and Algorithmic Fairness
• On Algorithmic Fairness and the EU Regulations