2021/09/03 by Riccardo Fogliato, Alexandra Chouldechova, Fogliato, Riccardo +3 · 1 citation
Computer Science · Psychology · Social Sciences · #Artificial Intelligence (cs.AI) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Human-Automation Interaction and Safety #Human-Computer Interaction (cs.HC) #Mobile Crowdsensing and Crowdsourcing
paper · pdf · doi:10.48550/arxiv.2109.01443
openalex publication_date 2021/09/03 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28
As algorithmic risk assessment instruments (RAIs) are increasingly adopted to\nassist decision makers, their predictive performance and potential to promote\ninequity have come under scrutiny. However, while most studies examine these\ntools in isolation, researchers have come to recognize that assessing their\nimpact requires understanding the behavior of their human interactants. In this\npaper, building off of several recent crowdsourcing works focused on criminal\njustice, we conduct a vignette study in which laypersons are tasked with\npredicting future re-arrests. Our key findings are as follows: (1) Participants\noften predict that an offender will be rearrested even when they deem the\nlikelihood of re-arrest to be well below 50%; (2) Participants do not anchor on\nthe RAI's predictions; (3) The time spent on the survey varies widely across\nparticipants and most cases are assessed in less than 10 seconds; (4) Judicial\ndecisions, unlike participants' predictions, depend in part on factors that are\northogonal to the likelihood of re-arrest. These results highlight the\ninfluence of several crucial but often overlooked design decisions and concerns\naround generalizability when constructing crowdsourcing studies to analyze the\nimpacts of RAIs.\n