Constrained Language Models Yield Few-Shot Semantic Parsers

2021/04/18 by Richard Shin, Shin, Richard, Christopher H. Lin +17 · 12 citations
Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2104.08768

EMNLP 2021. Code is available at https://github.com/microsoft/semantic_parsing_with_constrained_lm

arxiv created 2021/11/16 · arxiv updated 2021/11/18

Abstract

We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natural language. To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount of data and very little code to convert into English-like representations, our blueprint for rapidly bootstrapping semantic parsers leads to surprisingly effective performance on multiple community tasks, greatly exceeding baseline methods also trained on the same limited data.

Citations

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Cited by

• Large Language Models for Software Engineering Diagrams: A Systematic Review of UML and ER modelling
• STARE at the Structure: Steering ICL Exemplar Selection with Structural Alignment
• Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems
• Infherno: End-to-end Agent-based FHIR Resource Synthesis from Free-form Clinical Notes
• LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents
• Syntactic Control of Language Models by Posterior Inference
• Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
• Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation
• Earley-Driven Dynamic Pruning for Efficient Structured Decoding
• Suffix-Constrained Greedy Search Algorithms for Causal Language Models
• tAIfa: Enhancing Team Effectiveness and Cohesion with AI-Generated Automated Feedback
• Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling

Related

• Few-Shot Semantic Parsing with Language Models Trained On Code
• BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing
• On The Ingredients of an Effective Zero-shot Semantic Parser
• Training Naturalized Semantic Parsers with Very Little Data
• CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing