Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision

2016/10/31 by Chen Liang, Liang Chen, Jonathan Berant +9 · 1 voice · 44 citations
Computer Science · #Artificial intelligence #Artificial neural network #Computer science #Key (lock) #Knowledge base #Language model #Lisp #Machine learning #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Principle of compositionality #Process (computing) #Programmer #Programming language #Pruning #Sequence (biology) #Task (project management) #Theoretical computer science #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1611.00020

published in arXiv (Cornell University) (Cornell University) · ACL 2017 camera ready version

openalex publication_date 2016/10/31 · arxiv created 2017/04/23 · arxiv updated 2017/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base. In this work, we introduce a Neural Symbolic Machine, which contains (a) a neural "programmer", i.e., a sequence-to-sequence model that maps language utterances to programs and utilizes a key-variable memory to handle compositionality (b) a symbolic "computer", i.e., a Lisp interpreter that performs program execution, and helps find good programs by pruning the search space. We apply REINFORCE to directly optimize the task reward of this structured prediction problem. To train with weak supervision and improve the stability of REINFORCE, we augment it with an iterative maximum-likelihood training process. NSM outperforms the state-of-the-art on the WebQuestionsSP dataset when trained from question-answer pairs only, without requiring any feature engineering or domain-specific knowledge.

Citations

• Learning to Map Sentences to Logical Form: Structured Classification\n with Probabilistic Categorial Grammars
• Adam: A Method for Stochastic Optimization
• Compositional Semantic Parsing on Semi-Structured Tables
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• Neural GPUs Learn Algorithms
• Sequence Level Training with Recurrent Neural Networks
• Reinforcement Learning Neural Turing Machines - Revised
• Neural Programmer: Inducing Latent Programs with Gradient Descent
• Language to Logical Form with Neural Attention
• Glove: Global Vectors for Word Representation
• Mastering the game of Go with deep neural networks and tree search
• Deep Reinforcement Learning for Dialogue Generation
• Sequence-to-Sequence Learning as Beam-Search Optimization
• Reward Augmented Maximum Likelihood for Neural Structured Prediction
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Learning Dependency-Based Compositional Semantics

Cited by

• TreeQA: Enhanced LLM-RAG with logic tree reasoning for reliable and interpretable multi-hop question answering
• SEDM: Scalable Self-Evolving Distributed Memory for Agents
• PanelTR: Zero-Shot Table Reasoning Framework Through Multi-Agent Scientific Discussion
• Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)

Discussions

• Neural Symbolic Machines: Learning Semantic Parsers with Weak Supervision [hn, 94 points, 8 comments]

Related

• Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)
• Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning
• Large Language Models are Interpretable Learners
• Learning an Executable Neural Semantic Parser
• Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks