DataScout: Automatic Data Fact Retrieval for Statement Augmentation with an LLM-Based Agent

2025/04/24 by Yuqi Liu, Chen, Chuer, Liu, Yuqi +6
Business, Management and Accounting · Computer Science · Decision Sciences · #Big Data and Business Intelligence #Data Mining Algorithms and Applications #Data Quality and Management #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Information Retrieval (cs.IR)

paper · pdf · doi:10.48550/arxiv.2504.17334

openalex publication_date 2025/04/24 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

A data story typically integrates data facts from multiple perspectives and stances to construct a comprehensive and objective narrative. However, retrieving these facts demands time for data search and challenges the creator's analytical skills. In this work, we introduce DataScout, an interactive system that automatically performs reasoning and stance-based data facts retrieval to augment the user's statement. Particularly, DataScout leverages an LLM-based agent to construct a retrieval tree, enabling collaborative control of its expansion between users and the agent. The interface visualizes the retrieval tree as a mind map that eases users to intuitively steer the retrieval direction and effectively engage in reasoning and analysis. We evaluate the proposed system through case studies and in-depth expert interviews. Our evaluation demonstrates that DataScout can effectively retrieve multifaceted data facts from different stances, helping users verify their statements and enhance the credibility of their stories.

Citations

• SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledge
• Large Language Models for Information Retrieval: A Survey
• Is GPT-4 a Good Data Analyst?
• ChatGPT outperforms crowd workers for text-annotation tasks
• LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
• REPLUG: Retrieval-Augmented Black-Box Language Models
• Solo: Data Discovery Using Natural Language Questions Via A Self-Supervised Approach
• LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Improving language models by retrieving from trillions of tokens
• Talk2Data: A Natural Language Interface for Exploratory Visual Analysis via Question Decomposition
• Open Domain Question Answering over Tables via Dense Retrieval
• ChartStory: Automated Partitioning, Layout, and Captioning of Charts into Comic-Style Narratives
• Calliope: Automatic Visual Data Story Generation from a Spreadsheet
• NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries
• Language Models are Few-Shot Learners
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Task-Oriented Optimal Sequencing of Visualization Charts
• FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System
• Recent Trends in Deep Learning Based Natural Language Processing
• Self-Retrieval: End-to-End Information Retrieval with One Large Language Model

Related

• DataSage: Multi-agent Collaboration for Insight Discovery with External Knowledge Retrieval, Multi-role Debating, and Multi-path Reasoning
• Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking
• UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
• Fact-Checking with Contextual Narratives: Leveraging Retrieval-Augmented LLMs for Social Media Analysis
• Knowing When to Ask -- Bridging Large Language Models and Data