2024/05/07 by Yiming Lin, Lin, Yiming, Madelon Hulsebos +12 · 1 voice · 6 citations
Computer Science · #Analytics #Computer science #Data science #Information retrieval #Natural Language Processing Techniques #Natural language processing #Text and Document Classification Technologies #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2405.04674
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/05/07 · openalex created_date 2024/05/11 · openalex updated_date 2026/07/28
Unstructured data formats account for over 80% of the data currently stored, and extracting value from such formats remains a considerable challenge. In particular, current approaches for managing unstructured documents do not support ad-hoc analytical queries on document collections. Moreover, Large Language Models (LLMs) directly applied to the documents themselves, or on portions of documents through a process of Retrieval-Augmented Generation (RAG), fail to provide high accuracy query results, and in the LLM-only case, additionally incur high costs. Since many unstructured documents in a collection often follow similar templates that impart a common semantic structure, we introduce ZenDB, a document analytics system that leverages this semantic structure, coupled with LLMs, to answer ad-hoc SQL queries on document collections. ZenDB efficiently extracts semantic hierarchical structures from such templatized documents, and introduces a novel query engine that leverages these structures for accurate and cost-effective query execution. Users can impose a schema on their documents, and query it, all via SQL. Extensive experiments on three real-world document collections demonstrate ZenDB's benefits, achieving up to 30% cost savings compared to LLM-based baselines, while maintaining or improving accuracy, and surpassing RAG-based baselines by up to 61% in precision and 80% in recall, at a marginally higher cost.