2019/09/06 by Jorge Ramírez, Ramírez, Jorge, Simone Degiacomi +9
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Mobile Crowdsensing and Crowdsourcing #Open Source Software Innovations #Personal Information Management and User Behavior
paper · pdf · doi:10.48550/arxiv.1909.02800
openalex publication_date 2019/09/06 · openalex created_date 2019/09/12 · openalex updated_date 2026/07/28
We present CrowdHub, a tool for running systematic evaluations of task designs on top of crowdsourcing platforms. The goal is to support the evaluation process, avoiding potential experimental biases that, according to our empirical studies, can amount to 38% loss in the utility of the collected dataset in uncontrolled settings. Using CrowdHub, researchers can map their experimental design and automate the complex process of managing task execution over time while controlling for returning workers and crowd demographics, thus reducing bias, increasing utility of collected data, and making more efficient use of a limited pool of subjects.