2022/03/02 by Konstantinos Kogkalidis, Gijs Wijnholds, Kogkalidis, Konstantinos +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.2203.01063
8 pages plus references. To appear in Findings of the Association for Computational Linguistics 2022
openalex publication_date 2022/03/02 · arxiv created 2022/03/08 · arxiv updated 2022/03/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined.