2019/01/09 by Alex Kale, Matthew Kay, Kale, Alex +3 · 1 citation
Business, Management and Accounting · Computer Science · Decision Sciences · #Big Data and Business Intelligence #Data Visualization and Analytics #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Meta-analysis and systematic reviews
paper · pdf · doi:10.48550/arxiv.1901.02957
openalex publication_date 2019/01/09 · openalex created_date 2022/07/30 · openalex updated_date 2026/07/28
To make evidence-based recommendations to decision-makers, researchers\nconducting systematic reviews and meta-analyses must navigate a garden of\nforking paths: a series of analytical decision-points, each of which has the\npotential to influence findings. To identify challenges and opportunities\nrelated to designing systems to help researchers manage uncertainty around\nwhich of multiple analyses is best, we interviewed 11 professional researchers\nwho conduct research synthesis to inform decision-making within three\norganizations. We conducted a qualitative analysis identifying 480 analytical\ndecisions made by researchers throughout the scientific process. We present\ndescriptions of current practices in applied research synthesis and\ncorresponding design challenges: making it more feasible for researchers to try\nand compare analyses, shifting researchers' attention from rationales for\ndecisions to impacts on results, and supporting communication techniques that\nacknowledge decision-makers' aversions to uncertainty. We identify\nopportunities to design systems which help researchers explore, reason about,\nand communicate uncertainty in decision-making about possible analyses in\nresearch synthesis.\n