Learning Algebraic Recombination for Compositional Generalization

2021/07/14 by Chenyao Liu, Shengnan An, Liu, Chenyao +15
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2107.06516

openalex publication_date 2021/07/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Neural sequence models exhibit limited compositional generalization ability in semantic parsing tasks. Compositional generalization requires algebraic recombination, i.e., dynamically recombining structured expressions in a recursive manner. However, most previous studies mainly concentrate on recombining lexical units, which is an important but not sufficient part of algebraic recombination. In this paper, we propose LeAR, an end-to-end neural model to learn algebraic recombination for compositional generalization. The key insight is to model the semantic parsing task as a homomorphism between a latent syntactic algebra and a semantic algebra, thus encouraging algebraic recombination. Specifically, we learn two modules jointly: a Composer for producing latent syntax, and an Interpreter for assigning semantic operations. Experiments on two realistic and comprehensive compositional generalization benchmarks demonstrate the effectiveness of our model. The source code is publicly available at https://github.com/microsoft/ContextualSP.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
• Long Short-Term Memory
• Compositional Semantic Parsing on Semi-Structured Tables
• The Optimal Reward Baseline for Gradient-Based Reinforcement Learning
• The Evolved Transformer
• Neural Shuffle-Exchange Networks -- Sequence Processing in O(n log n) Time
• Compositional Generalization for Primitive Substitutions
• Measuring Compositional Generalization: A Comprehensive Method on Realistic Data
• Compositional Generalization by Learning Analytical Expressions
• Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
• COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
• Sequence-Level Mixed Sample Data Augmentation

Related

• Improving Compositional Generalization with Latent Structure and Data Augmentation
• Compositional Generalization via Neural-Symbolic Stack Machines
• Compositional Generalization by Learning Analytical Expressions
• Neural-Symbolic Recursive Machine for Systematic Generalization
• Learning to Recombine and Resample Data for Compositional Generalization