2025/06/19 by Maeve Hutchinson, Radu Jianu, Hutchinson, Maeve +7
Computer Science · #Computation and Language (cs.CL) #Data Visualization and Analytics #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)
paper · doi:10.48550/arxiv.2506.16571
openalex publication_date 2025/06/19 · openalex created_date 2025/10/09 · openalex updated_date 2026/07/29
Prior natural language datasets for data visualization have focused on tasks such as visualization literacy assessment, insight generation, and visualization generation from natural language instructions. These studies often rely on controlled setups with purpose-built visualizations and artificially constructed questions. As a result, they tend to prioritize the interpretation of visualizations, focusing on decoding visualizations rather than understanding their encoding. In this paper, we present a new dataset and methodology for probing visualization design rationale through natural language. We leverage a unique source of real-world visualizations and natural language narratives: literate visualization notebooks created by students as part of a data visualization course. These notebooks combine visual artifacts with design exposition, in which students make explicit the rationale behind their design decisions. We also use large language models (LLMs) to generate and categorize question-answer-rationale triples from the narratives and articulations in the notebooks. We then carefully validate the triples and curate a dataset that captures and distills the visualization design choices and corresponding rationales of the students.