2019/08/26 by Geoff Bacon, Bacon, Geoff, Terry Regier +1 · 8 citations
Computer Science · #Agreement #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Epistemology #FOS: Computer and information sciences #Language structure #Linguistics #Natural Language Processing Techniques #Natural language processing #Phenomenon #Philosophy #Physics #Programming language #Semantics (computer science) #Syntactic structure #Syntax #Text Readability and Simplification #Topic Modeling #cs.CL
paper · pdf · doi:10.48550/arxiv.1908.09892
published in arXiv (Cornell University) (Cornell University) · 6 pages
arxiv created 2019/08/26 · openalex publication_date 2019/08/26 · arxiv updated 2019/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
Learning representations that accurately model semantics is an important goal\nof natural language processing research. Many semantic phenomena depend on\nsyntactic structure. Recent work examines the extent to which state-of-the-art\nmodels for pre-training representations, such as BERT, capture such\nstructure-dependent phenomena, but is largely restricted to one phenomenon in\nEnglish: number agreement between subjects and verbs. We evaluate BERT's\nsensitivity to four types of structure-dependent agreement relations in a new\nsemi-automatically curated dataset across 26 languages. We show that both the\nsingle-language and multilingual BERT models capture syntax-sensitive agreement\npatterns well in general, but we also highlight the specific linguistic\ncontexts in which their performance degrades.\n