GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

2020/09/29 by Tao Yu, Chien-Sheng Wu, Yu, Tao +15 · 12 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text and Document Classification Technologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2009.13845

openalex publication_date 2020/09/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data. We construct synthetic question-SQL pairs over high-quality tables via a synchronous context-free grammar (SCFG) induced from existing text-to-SQL datasets. We pre-train our model on the synthetic data using a novel text-schema linking objective that predicts the syntactic role of a table field in the SQL for each question-SQL pair. To maintain the model's ability to represent real-world data, we also include masked language modeling (MLM) over several existing table-and-language datasets to regularize the pre-training process. On four popular fully supervised and weakly supervised table semantic parsing benchmarks, GraPPa significantly outperforms RoBERTa-large as the feature representation layers and establishes new state-of-the-art results on all of them.

Citations

• Compositional Semantic Parsing on Semi-Structured Tables
• A Comprehensive Exploration on WikiSQL with Table-Aware Word\n Contextualization
• Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation
• TabFact: A Large-scale Dataset for Table-based Fact Verification
• A Discrete Hard EM Approach for Weakly Supervised Question Answering
• HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data
• Pre-training via Paraphrasing

Cited by

• Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQL
• Schema-Aware Localisation (SAL): Live Schema Grounding and Hallucination Validation for Oracle NL2SQL
• An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data
• Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQL
• Questionnaire meets LLM: A Benchmark and Empirical Study of Structural Skills for Understanding Questions and Responses
• KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers
• Relation Aware Semi-autoregressive Semantic Parsing for NL2SQL
• K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling
• Development of Automated Software Design Document Review Methods Using Large Language Models
• Evaluating NL2SQL via SQL2NL
• PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussion
• ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser

Related

• GP: Context-free Grammar Pre-training for Text-to-SQL Parsers
• Linking-Enhanced Pre-Training for Table Semantic Parsing
• Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training
• TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
• STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing