Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL

2024/02/16 by Dingzirui Wang, Wang, Dingzirui, Longxu Dou +7 · 2 citations
Computer Science · Decision Sciences · #Advanced Database Systems and Queries #Cloud Computing and Resource Management #Scientific Computing and Data Management

paper · pdf · doi:10.48550/arxiv.2402.10663

Abstract

Currently, the in-context learning method based on large language models (LLMs) has become the mainstream of text-to-SQL research. Previous works have discussed how to select demonstrations related to the user question from a human-labeled demonstration pool. However, human labeling suffers from the limitations of insufficient diversity and high labeling overhead. Therefore, in this paper, we discuss how to measure and improve the diversity of the demonstrations for text-to-SQL. We present a metric to measure the diversity of the demonstrations and analyze the insufficient of the existing labeled data by experiments. Based on the above discovery, we propose fusing iteratively for demonstrations (Fused) to build a high-diversity demonstration pool through human-free multiple-iteration synthesis, improving diversity and lowering label cost. Our method achieves an average improvement of 3.2% and 5.0% with and without human labeling on several mainstream datasets, which proves the effectiveness of Fused.

Cited by

• Skeletons Matter: Dynamic Data Augmentation for Text-to-Query
• ORANGE: An Online Reflection ANd GEneration framework with Domain Knowledge for Text-to-SQL

Related

• Selective Demonstrations for Cross-domain Text-to-SQL
• Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies
• MageSQL: Enhancing In-context Learning for Text-to-SQL Applications with Large Language Models
• Open-SQL Framework: Enhancing Text-to-SQL on Open-source Large Language Models
• SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data