Causal Analysis of Syntactic Agreement Mechanisms in Neural Language\n Models

2021/06/10 by Matthew Finlayson, Finlayson, Matthew, Aaron Mueller +9 · 15 citations
Computer Science · #68T50 #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7 #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2106.06087

openalex publication_date 2021/06/10 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

Targeted syntactic evaluations have demonstrated the ability of language\nmodels to perform subject-verb agreement given difficult contexts. To elucidate\nthe mechanisms by which the models accomplish this behavior, this study applies\ncausal mediation analysis to pre-trained neural language models. We investigate\nthe magnitude of models' preferences for grammatical inflections, as well as\nwhether neurons process subject-verb agreement similarly across sentences with\ndifferent syntactic structures. We uncover similarities and differences across\narchitectures and model sizes -- notably, that larger models do not necessarily\nlearn stronger preferences. We also observe two distinct mechanisms for\nproducing subject-verb agreement depending on the syntactic structure of the\ninput sentence. Finally, we find that language models rely on similar sets of\nneurons when given sentences with similar syntactic structure.\n

Cited by

• A Mechanistic Perspective and Difficulty Metric for Unlearning
• Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
• On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception
• PAHQ: Accelerating Automated Circuit Discovery through Mixed-Precision Inference Optimization
• Diagnosing Bottlenecks in Data Visualization Understanding by Vision-Language Models
• BlackboxNLP-2025 MIB Shared Task: Exploring Ensemble Strategies for Circuit Localization Methods
• Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context
• Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
• Steering Prepositional Phrases in Language Models: A Case of with-headed Adjectival and Adverbial Complements in Gemma-2
• Distributed Specialization: Rare-Token Neurons in Large Language Models
• Evolution of Concepts in Language Model Pre-Training
• RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching
• Detecting and Characterizing Planning in Language Models
• The Cultural Gene of Large Language Models: A Study on the Impact of Cross-Corpus Training on Model Values and Biases
• How Causal Abstraction Underpins Computational Explanation

Related

• Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models
• Different types of syntactic agreement recruit the same units within large language models
• Discourse structure interacts with reference but not syntax in neural language models
• Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages
• Neural Language Models as Psycholinguistic Subjects: Representations of\n Syntactic State