2020/04/04 by Bhavya Ghai, Ghai, Bhavya, Q. Vera Liao +5
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Misinformation and Its Impacts #Mobile Crowdsensing and Crowdsourcing
paper · pdf · doi:10.48550/arxiv.2004.02028
openalex publication_date 2020/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Social biases based on gender, race, etc. have been shown to pollute machine learning (ML) pipeline predominantly via biased training datasets. Crowdsourcing, a popular cost-effective measure to gather labeled training datasets, is not immune to the inherent social biases of crowd workers. To ensure such social biases aren't passed onto the curated datasets, it's important to know how biased each crowd worker is. In this work, we propose a new method based on counterfactual fairness to quantify the degree of inherent social bias in each crowd worker. This extra information can be leveraged together with individual worker responses to curate a less biased dataset.