2025/09/18 by Kunpeng Ai, Haichao Miao, Ai, Kuangshi +7 · 1 citation
Computer Science · #Computation and Language (cs.CL) #Data Visualization and Analytics #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Graphics (cs.GR) #Human-Computer Interaction (cs.HC) #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.2509.15160
openalex publication_date 2025/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent advances in multi-modal large language models (MLLMs) have enabled increasingly sophisticated autonomous visualization agents capable of translating user intentions into data visualizations. However, measuring progress and comparing different agents remains challenging, particularly in scientific visualization (SciVis), due to the absence of comprehensive, large-scale benchmarks for evaluating real-world capabilities. This position paper examines the various types of evaluation required for SciVis agents, outlines the associated challenges, provides a simple proof-of-concept evaluation example, and discusses how evaluation benchmarks can facilitate agent self-improvement. We advocate for a broader collaboration to develop a SciVis agentic evaluation benchmark that would not only assess existing capabilities but also drive innovation and stimulate future development in the field.