2019/11/27 by Chee Wee Leong, Leong, Chee Wee, Katrina Crotts Roohr +15
Computer Science · Neuroscience · Psychology · Social Sciences · #Adversarial Robustness in Machine Learning #Deception detection and forensic psychology #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #H.1.2 #Human-Computer Interaction (cs.HC) #I.2.0 #Machine Learning (cs.LG) #Psychology of Moral and Emotional Judgment #Social and Intergroup Psychology
paper · pdf · doi:10.48550/arxiv.1911.13248
openalex publication_date 2019/11/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Supervised systems require human labels for training. But, are humans themselves always impartial during the annotation process? We examine this question in the context of automated assessment of human behavioral tasks. Specifically, we investigate whether human ratings themselves can be trusted at their face value when scoring video-based structured interviews, and whether such ratings can impact machine learning models that use them as training data. We present preliminary empirical evidence that indicates there might be biases in such annotations, most of which are visual in nature.