European Union regulations on algorithmic decision-making and a "right to explanation"
2016/06/28 by Bryce Goodman, Seth Flaxman · 10 voices · 2,106 citations
Computer Science · Mathematics · Medicine · Social Sciences · #Artificial Intelligence in Healthcare and Education #Business #Computer science #Economics #Ethics and Social Impacts of AI #European union #International trade #Law and economics #Law, AI, and Intellectual Property #Management science #Political science #Sociology #cs.CY #cs.LG #stat.ML
paper · pdf · open access · doi:10.1609/aimag.v38i3.2741
published in AI Magazine 38(3), 50-57 (Association for the Advancement of Artificial Intelligence) · presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY
arxiv created 2016/08/31 · openalex publication_date 2017/09/01 · arxiv updated 2018/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Abstract
We summarize the potential impact that the European Union's new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which "significantly affect" users. The law will also effectively create a "right to explanation," whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.
Cited by
- Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications
- Relation between prognostics predictor evaluation metrics and local interpretability SHAP values
- Balancing explainability and privacy in AI systems: A strategic imperative for managers
- Toward Human-Understandable, Explainable AI
- A Unifying Review of Deep and Shallow Anomaly Detection
- Transparency in Complex Computational Systems
- Artificial intelligence in orthopaedics surgery: transforming technological innovation in patient care and surgical training
- COVAriance-Induced Fairness Gap Penalty for Subgroup-Fair Clustering
- Footprints of Data in a Classifier: Understanding the Privacy Risks and Solution Strategies
- Machine Learning Testing: Survey, Landscapes and Horizons
- GANMEX: One-vs-One Attributions Guided by GAN-based Counterfactual Explanation Baselines
- Computing Optimal Decision Sets with SAT
- Extracting spatial effects from machine learning model using local interpretation method: An example of SHAP and XGBoost
- Information Theoretic Measures for Fairness-aware Feature Selection
- Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisation
- Concept-based Visual Counterfactual Explanations with Diffusion Models
- Scalable Explanation of Inferences on Large Graphs
- Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition
- Imputation Uncertainty in Interpretable Machine Learning Methods
- A Framework for Causal Concept-based Model Explanations
- CryptoSPN: Privacy-preserving Sum-Product Network Inference
- GDPR Anti-Patterns: How Design and Operation of Modern Cloud-scale Systems Conflict with GDPR
- Bias Amplification in Artificial Intelligence Systems
- The effectiveness of feature attribution methods and its correlation with automatic evaluation scores
- Interpretable Deep Convolutional Fuzzy Classifier
- SecureBoost: A Lossless Federated Learning Framework
- The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI
- Characterizing Intersectional Group Fairness with Worst-Case Comparisons
- State of the Art in Fair ML: From Moral Philosophy and Legislation to\n Fair Classifiers
- Evaluating neural network explanation methods using hybrid documents and morphological agreement
- Trustworthy AI: From Principles to Practices
- Scientific Exploration and Explainable Artificial Intelligence
- The role of artificial intelligence for management decision: a structured literature review
- The Need for Standardized Explainability
- A Sparsity Algorithm with Applications to Corporate Credit Rating
- Discrimination, intelligence artificielle et decisions algorithmiques
- Discrimination, artificial intelligence, and algorithmic decision-making
- Exact and Consistent Interpretation of Piecewise Linear Models Hidden behind APIs: A Closed Form Solution
- From black box to clarity: Strategies for effective AI informed consent in healthcare
- The Answer Lies Within: Self-Derived Rewards Enable Explainable Relation Extraction
- Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing
- Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
- Structural Refinement of Bayesian Networks for Efficient Model Parameterisation
- A systematic review of algorithm aversion in augmented decision making
- Black-box artificial intelligence: an epistemological and critical analysis
- CausalKANs: interpretable treatment effect estimation with Kolmogorov-Arnold networks
- Learning Conformal Explainers for Image Classifiers
- Adaptive Cybersecurity Architecture for Digital Product Ecosystems Using Agentic AI
- Training Feedforward Neural Networks with Standard Logistic Activations is Feasible
- Better together? Human oversight as means to achieve fairness in the European AI Act governance
- Interpretability of neural networks via polynomial representations
- Posthoc Interpretability of Learning to Rank Models using Secondary Training Data
- Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
- BMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability
- Truthful AI: Developing and governing AI that does not lie
- Transparent and Fair Profiling in Employment Services: Evidence from Switzerland
- Progressive Disclosure: Designing for Effective Transparency
- Minimal Data, Maximum Clarity: A Heuristic for Explaining Optimization
- Trustworthy Artificial Intelligence: A Review
- Explainable AI and Adoption of Financial Algorithmic Advisors: an Experimental Study
- Evaluating the stability of model explanations in instance-dependent cost-sensitive credit scoring
- Deep opacity and AI: A threat to XAI and to privacy protection mechanisms
- Extending LIME for Business Process Automation
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- Detecting Statistical Interactions from Neural Network Weights
- AI and Agile Software Development: A Research Roadmap from the XP2025 Workshop
- Transparency, Auditability and eXplainability of Machine Learning Models\n in Credit Scoring
- Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
- Automated Decision-Making and the Right to an Explanation Under POPIA in South Africa: A Legal Perspective
- An interpretable semi-supervised classifier using two different strategies for amended self-labeling
- A Novel Study on Intelligent Methods and Explainable AI for Dynamic Malware Analysis
- Towards Interpretable Deep Extreme Multi-label Learning
- Collective dynamics of strategic classification
- Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models
- Fairwashing: the risk of rationalization
- Discovery of Important Subsequences in Electrocardiogram Beats Using the Nearest Neighbour Algorithm
- Interpreting Classifiers through Attribute Interactions in Datasets
- Feature Selection in Nonlinear SVMs via Local Search and Submodular Optimization
- Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling
- Boosting Team Modeling through Tempo-Relational Representation Learning
- Self-learn to Explain Siamese Networks Robustly
- A copula-based visualization technique for a neural network
- An Overview on the Landscape of R Packages for Credit Scoring
- Outlining the Design Space of Explainable Intelligent Systems for Medical Diagnosis
- Understanding Black-box Predictions via Influence Functions
- Machine Learning Testing: Survey, Landscapes and Horizons
- Designing User-Centric Metrics for Evaluation of Counterfactual Explanations
- Ethical Considerations in Artificial Intelligence Courses
- Metrics for Explainable AI: Challenges and Prospects
- Incorporating Priors with Feature Attribution on Text Classification
- Beyond Fairness Metrics: Roadblocks and Challenges for Ethical AI in Practice
- Towards A Rigorous Science of Interpretable Machine Learning
- Reasoning on Knowledge Graphs with Debate Dynamics
- Teaching Responsible Data Science: Charting New Pedagogical Territory
- Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning
- The Mythos of Model Interpretability
- Chimeric U-Net – Modifying the standard U-Net towards explainability
- Unexplainability and Incomprehensibility of Artificial Intelligence
- The Doctor Just Won't Accept That!
- Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
- A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanations
- Beyond the ATE: Interpretable Modelling of Treatment Effects over Dose and Time
- On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classification
- Bias in Machine Learning -- What is it Good for?
- A First Look: Towards Explainable TextVQA Models via Visual and Textual Explanations
- A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
- The Seven Sins of Personal-Data Processing Systems under GDPR
- Evaluating explainable AI for deep learning-based network intrusion detection system alert classification
- Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning: A Study on a Swedish Cohort
- Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
- On Interpretability of Artificial Neural Networks: A Survey
- Transparent Classification with Multilayer Logical Perceptrons and Random Binarization
- CACTUS as a Reliable Tool for Early Classification of Age-related Macular Degeneration
- Model extraction from counterfactual explanations
- Achieving Transparency Report Privacy in Linear Time
- Debate Dynamics for Human-comprehensible Fact-checking on Knowledge Graphs
- Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases
- Optimal Cooperative Inference
- Prediction via Shapley Value Regression
- Optimal Piecewise Local-Linear Approximations
- RuleMatrix: Visualizing and Understanding Classifiers with Rules
- Towards interpreting ML-based automated malware detection models: a survey
- MeLIME: Meaningful Local Explanation for Machine Learning Models
- Personalized prediction of daily eczema severity scores using a mechanistic machine learning model
- Feature Relevancy, Necessity and Usefulness: Complexity and Algorithms
- Multi-criteria Rank-based Aggregation for Explainable AI
- COSMIC: Generalized Refusal Direction Identification in LLM Activations
- Individualised Counterfactual Examples Using Conformal Prediction Intervals
- Artificial Intelligence and the Future of Work: A Proactive Strategy
- MultiPhishGuard: An LLM-based Multi-Agent System for Phishing Email Detection
- Principles of Explanation in Human-AI Systems
- Now You See Me (CME): Concept-based Model Extraction
- Providing Explanations for Recommendations in Reciprocal Environments
- Learning Small Decision Trees with Few Outliers: A Parameterized Perspective
- Using Artificial Intelligence to classify Jobseekers: The Accuracy-Equity Trade-off
- Bioinformatics and Medicine in the Era of Deep Learning
- Quantifying contributions of geographical features to urban GDP outputs via interpretable machine learning
- On the definition and importance of interpretability in scientific machine learning
- Artificial intelligence in radiation oncology
- From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices
- Artificial Intelligence and Modeling & Simulation: An Overview
- Safe Handover in Mixed-Initiative Control for Cyber-Physical Systems
- Towards Explainable Fact Checking
- Hows and Whys of Artificial Intelligence for Public Sector Decisions: Explanation and Evaluation
- Securing the Future of IVR: AI-Driven Innovation with Agile Security, Data Regulation, and Ethical AI Integration
- Thinking Outside the Template with Modular GP-GOMEA
- Designing AI for Explainability and Verifiability: A Value Sensitive Design Approach to Avoid Artificial Stupidity in Autonomous Vehicles
- Automation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World
- Embodied Explainability and Ontological Obstacles: Why We Struggle to Explain the Answers of Large Language Models (LLMs)
- Assessing the Local Interpretability of Machine Learning Models
- Towards Intelligent Regulation of Artificial Intelligence
- Definitions, methods, and applications in interpretable machine learning
- Towards Improved Cervical Cancer Screening: Vision Transformer-Based Classification and Interpretability
- Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities
- Arming the public with artificial intelligence to counter social bots
- The Bouncer Problem: Challenges to Remote Explainability
- Device-Native Autonomous Agents for Privacy-Preserving Negotiations
- Learning Fair Rule Lists
- Implementations in Machine Ethics
- Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models
- Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room
- Preventing the Generation of Inconsistent Sets of Classification Rules
- Transformation of audio embeddings into interpretable, concept-based representations
- An Overview of Computational Approaches for Interpretation Analysis
- Food fraud detection using explainable artificial intelligence
- Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach
- A Self-Explainable Deep Architecture for Security Applications
- Why am I seeing this? Towards recognizing social media recommender systems with missing recommendations
- Automated assessment of psychiatric disorders using speech: A systematic review
- Usability Testing of an Explainable AI-enhanced Tool for Clinical Decision Support: Insights from the Reflexive Thematic Analysis
- Artificial intelligence [wikipedia]
Discussions
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 104 points, 120 comments]
- EU regulations on algorithmic decision-making and a "right to explanation" [lobsters, 21 points, 11 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 4 points, 0 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 2 points, 0 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 2 points, 0 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 2 points, 0 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 1 points, 0 comments]
- EU regulations on algorithmic decision-making and a “right to explanation” [hn, 1 points, 0 comments]
- EU law to go into effect in 2018 will establish "right to explanation" for machine learning based decisions. PDF: https://arxiv.org/pdf/1606.08813v1.pdf [bsky, 0 points, 0 comments]
- "EU regulations on algorithmic decision-making and a 'right to explanation'", by Bryce Goodman and Seth Flaxman. http://arxiv.org/pdf/1606.08813v1.pdf [bsky, 0 points, 0 comments]
Related