Improving Compositional Generalization in Classification Tasks via Structure Annotations

2021/06/19 by Kim, Juyong, Ravikumar, Pradeep, Ainslie, Joshua +1
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2106.10434

Abstract

Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. Although humans seem to have a great ability to generalize compositionally, state-of-the-art neural models struggle to do so. In this work, we study compositional generalization in classification tasks and present two main contributions. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.

Related

• Improving Compositional Generalization with Latent Structure and Data Augmentation
• Concepts, Properties and an Approach for Compositional Generalization
• Analyzing the Inner Workings of Transformers in Compositional Generalization
• Compositional Generalization Requires Compositional Parsers
• Data Factors for Better Compositional Generalization