2022/01/01 by Arkil Patel, Satwik Bhattamishra, Phil Blunsom +1 · 1 voice
Arts and Humanities · Health Professions · #Aging, Elder Care, and Social Issues #Health, Medicine and Society #Hermeneutics and Narrative Identity
paper · pdf · doi:10.18653/v1/2022.acl-short.46
openalex publication_date 2022/01/01 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/29
Compositional generalization is a fundamental trait in humans, allowing us to effortlessly combine known phrases to form novel sentences. Recent works have claimed that standard seqto-seq models severely lack the ability to compositionally generalize. In this paper, we focus on one-shot primitive generalization as introduced by the popular SCAN benchmark. We demonstrate that modifying the training distribution in simple and intuitive ways enables standard seq-to-seq models to achieve nearperfect generalization performance, thereby showing that their compositional generalization abilities were previously underestimated. We perform detailed empirical analysis of this phenomenon. Our results indicate that the generalization performance of models is highly sensitive to the characteristics of the training data which should be carefully considered while designing such benchmarks in future.