Evaluating Induced CCG Parsers on Grounded Semantic Parsing

2016/09/29 by Bisk, Yonatan, Reddy, Siva, Blitzer, John +2
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1609.09405

Abstract

We compare the effectiveness of four different syntactic CCG parsers for a semantic slot-filling task to explore how much syntactic supervision is required for downstream semantic analysis. This extrinsic, task-based evaluation provides a unique window to explore the strengths and weaknesses of semantics captured by unsupervised grammar induction systems. We release a new Freebase semantic parsing dataset called SPADES (Semantic PArsing of DEclarative Sentences) containing 93K cloze-style questions paired with answers. We evaluate all our models on this dataset. Our code and data are available at https://github.com/sivareddyg/graph-parser.

Related

• CCG Parsing and Multiword Expressions
• Automatic Generation of High Quality CCGbanks for Parser Domain Adaptation
• ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation
• COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
• BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing