On Adversarial Bias and the Robustness of Fair Machine Learning

2020/06/15 by Hongyan Chang, Ta Duy Nguyen, Chang, Hongyan +7 · 1 citation
Computer Science · Social Sciences · #Adversarial Robustness in Machine Learning #Computers and Society (cs.CY) #Cryptography and Security (cs.CR) #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2006.08669

openalex publication_date 2020/06/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Optimizing prediction accuracy can come at the expense of fairness. Towards minimizing discrimination against a group, fair machine learning algorithms strive to equalize the behavior of a model across different groups, by imposing a fairness constraint on models. However, we show that giving the same importance to groups of different sizes and distributions, to counteract the effect of bias in training data, can be in conflict with robustness. We analyze data poisoning attacks against group-based fair machine learning, with the focus on equalized odds. An adversary who can control sampling or labeling for a fraction of training data, can reduce the test accuracy significantly beyond what he can achieve on unconstrained models. Adversarial sampling and adversarial labeling attacks can also worsen the model's fairness gap on test data, even though the model satisfies the fairness constraint on training data. We analyze the robustness of fair machine learning through an empirical evaluation of attacks on multiple algorithms and benchmark datasets.

Citations

• Equality of Opportunity in Supervised Learning
• Understanding Black-box Predictions via Influence Functions
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Counterfactual Fairness
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Learning Adversarially Fair and Transferable Representations
• A Reductions Approach to Fair Classification
• Identifying and Correcting Label Bias in Machine Learning

Cited by

• Covert Attacks on Machine Learning Training in Passively Secure MPC

Related

• Poisoning Attacks on Fair Machine Learning
• Adversarial Bias: Data Poisoning Attacks on Fairness
• Poisoning Attacks on Algorithmic Fairness
• To be Robust or to be Fair: Towards Fairness in Adversarial Training
• Fairness-aware Regression Robust to Adversarial Attacks