Automatically Identifying Local and Global Circuits with Linear Computation Graphs

2024/05/22 by Xuyang Ge, Fukang Zhu, Ge, Xuyang +9 · 10 citations
Computer Science · Engineering · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Low-power high-performance VLSI design #Machine Learning (cs.LG) #Quantum Computing Algorithms and Architecture #VLSI and FPGA Design Techniques

paper · pdf · doi:10.48550/arxiv.2405.13868

openalex publication_date 2024/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Circuit analysis of any certain model behavior is a central task in mechanistic interpretability. We introduce our circuit discovery pipeline with Sparse Autoencoders (SAEs) and a variant called Transcoders. With these two modules inserted into the model, the model's computation graph with respect to OV and MLP circuits becomes strictly linear. Our methods do not require linear approximation to compute the causal effect of each node. This fine-grained graph identifies both end-to-end and local circuits accounting for either logits or intermediate features. We can scalably apply this pipeline with a technique called Hierarchical Attribution. We analyze three kinds of circuits in GPT-2 Small: bracket, induction, and Indirect Object Identification circuits. Our results reveal new findings underlying existing discoveries.

Cited by

• Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
• Circuit Insights: Towards Interpretability Beyond Activations
• Evolution of Concepts in Language Model Pre-Training
• Attention Layers Add Into Low-Dimensional Residual Subspaces
• Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning
• Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
• K-MSHC: Unmasking Minimally Sufficient Head Circuits in Large Language Models with Experiments on Syntactic Classification Tasks
• Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
• Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decomposition
• Towards Combinatorial Interpretability of Neural Computation

Related

• Local Causal Discovery for Estimating Causal Effects
• Latent Confounded Causal Discovery via Lie Bracket Geometry
• LoSAM: Local Search in Additive Noise Models with Mixed Mechanisms and General Noise for Global Causal Discovery
• Causal Structural Learning Via Local Graphs
• Hybrid Top-Down Global Causal Discovery with Local Search for Linear and Nonlinear Additive Noise Models