TableRAG: Million-Token Table Understanding with Language Models

2024/10/07 by Sian Chen, Si-An Chen, Lesly Miculicich +18 · 2 voices · 15 citations
Computer Science · Decision Sciences · #Data Quality and Management #Handwritten Text Recognition Techniques #Natural Language Processing Techniques #cs.AI #cs.CL #cs.IR #cs.LG

paper · pdf · doi:10.48550/arxiv.2410.04739

arxiv published 2024/10/07 · arxiv updated 2024/12/26

Abstract

Recent advancements in language models (LMs) have notably enhanced their ability to reason with tabular data, primarily through program-aided mechanisms that manipulate and analyze tables. However, these methods often require the entire table as input, leading to scalability challenges due to the positional bias or context length constraints. In response to these challenges, we introduce TableRAG, a Retrieval-Augmented Generation (RAG) framework specifically designed for LM-based table understanding. TableRAG leverages query expansion combined with schema and cell retrieval to pinpoint crucial information before providing it to the LMs. This enables more efficient data encoding and precise retrieval, significantly reducing prompt lengths and mitigating information loss. We have developed two new million-token benchmarks from the Arcade and BIRD-SQL datasets to thoroughly evaluate TableRAG's effectiveness at scale. Our results demonstrate that TableRAG's retrieval design achieves the highest retrieval quality, leading to the new state-of-the-art performance on large-scale table understanding.

Cited by

• Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering
• JT-DA: Enhancing Data Analysis with Tool-Integrated Table Reasoning Large Language Models
• Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization
• STaR: Towards Cognitive Table Reasoning via Slow-Thinking Large Language Models
• NOTAM-Evolve: A Knowledge-Guided Self-Evolving Optimization Framework with LLMs for NOTAM Interpretation
• How Good Are LLMs at Processing Tool Outputs?
• Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation
• AutoPK: Leveraging LLMs and a Hybrid Similarity Metric for Advanced Retrieval of Pharmacokinetic Data from Complex Tables and Documents
• Same Content, Different Representations: A Controlled Study for Table QA
• Scan-do Attitude: Towards Autonomous CT Protocol Management using a Large Language Model Agent
• When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables
• Improving Table Understanding with LLMs and Entity-Oriented Search
• TableZoomer: A Collaborative Agent Framework for Large-scale Table Question Answering
• Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaboration
• Improving Table Retrieval with Question Generation from Partial Tables

Discussions

• TableRAG: Million-Token Table Understanding with Language Models [hn, 2 points, 0 comments]
• TableRAG: Million-Token Table Understanding with Language Models [hn, 1 points, 0 comments]

Related

• TabRAG: Tabular Document Retrieval via Structured Language Representations
• Table as a Modality for Large Language Models
• RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking
• Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study
• TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning