Towards Automated Circuit Discovery for Mechanistic Interpretability

2023/04/28 by Arthur Conmy, Augustine N. Mavor-Parker, Conmy, Arthur +7 · 76 citations
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2304.14997

openalex publication_date 2023/04/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Through considerable effort and intuition, several recent works have reverse-engineered nontrivial behaviors of transformer models. This paper systematizes the mechanistic interpretability process they followed. First, researchers choose a metric and dataset that elicit the desired model behavior. Then, they apply activation patching to find which abstract neural network units are involved in the behavior. By varying the dataset, metric, and units under investigation, researchers can understand the functionality of each component. We automate one of the process' steps: to identify the circuit that implements the specified behavior in the model's computational graph. We propose several algorithms and reproduce previous interpretability results to validate them. For example, the ACDC algorithm rediscovered 5/5 of the component types in a circuit in GPT-2 Small that computes the Greater-Than operation. ACDC selected 68 of the 32,000 edges in GPT-2 Small, all of which were manually found by previous work. Our code is available at https://github.com/ArthurConmy/Automatic-Circuit-Discovery.

Cited by

• Grounding latent algorithm routing in transformer reasoning
• Mechanistic Analysis of Circuit Preservation in Federated Learning
• Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models
• A Mechanistic Perspective and Difficulty Metric for Unlearning
• Secret mixtures of experts inside your LLM
• The Deleuzian Representation Hypothesis
• Multi-Granular Node Pruning for Causal Circuit Discovery
• Intrinsically Interpretable Attention via Sparse Post-Training
• Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity
• Mapping Clinical Doubt: Locating Linguistic Uncertainty in LLMs
• Emergence and Localisation of Semantic Role Circuits in LLMs
• Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
• Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks
• Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models
• Anatomy of an Idiom: Tracing Non-Compositionality in Language Models
• Beyond Tokens in Language Models: Interpreting Activations through Text Genre Chunks
• nnterp: A Standardized Interface for Mechanistic Interpretability of Transformers
• Weight-sparse transformers have interpretable circuits
• Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning
• Which Sparse Autoencoder Features Are Real? Model-X Knockoffs for False Discovery Rate Control
• SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMs
• APP: Accelerated Path Patching with Task-Specific Pruning
• Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI
• Towards Verifiable Transformers: Solver-Checkable Circuit Explanations
• Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers
• BlackboxNLP-2025 MIB Shared Task: Improving Circuit Faithfulness via Better Edge Selection
• PAHQ: Accelerating Automated Circuit Discovery through Mixed-Precision Inference Optimization
• Rule-Based Explanations for Retrieval-Augmented LLM Systems
• Mapping Faithful Reasoning in Language Models
• Training-Free Spectral Fingerprints of Voice Processing in Transformers
• C-SWAP: Explainability-Aware Structured Pruning for Efficient Neural Networks Compression
• Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
• Circuit Insights: Towards Interpretability Beyond Activations
• Programmatic Representation Learning with Language Models
• CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions
• The Curious Case of Factual (Mis)Alignment between LLMs' Short- and Long-Form Answers
• QLENS: Towards A Quantum Perspective of Language Transformers
• Discursive Circuits: How Do Language Models Understand Discourse Relations?
• Cluster Paths: Navigating Interpretability in Neural Networks
• Agentic Misalignment: How LLMs Could Be Insider Threats
• Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling
• Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
• ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack
• EXP-CAM: Explanation Generation and Circuit Discovery Using Classifier Activation Matching
• Interpret, prune and distill Donut : towards lightweight VLMs for VQA on document
• Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Training
• Circuit Distillation
• Query Circuits: Explaining How Language Models Answer User Prompts
• Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
• Uncovering Grounding IDs: How External Cues Shape Multimodal Binding
• Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research
• REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model
• Towards Transparent AI: A Survey on Explainable Language Models
• Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs
• CLUE: Conflict-guided Localization for LLM Unlearning Framework
• A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering
• Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
• Gaze Heads: How VLMs Look at What They Describe
• Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Models
• Achilles' Heel of Mamba: Essential difficulties of the Mamba architecture demonstrated by synthetic data
• Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research
• Programmable Cognitive Bias in Social Agents
• Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content
• Pathological Truth Bias in Vision-Language Models
• Transparency of medical artificial intelligence systems
• Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining
• From Indirect Object Identification to Syllogisms: Exploring Binary Mechanisms in Transformer Circuits
• RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching
• Robustness is Important: Limitations of LLMs for Data Fitting
• Even Heads Fix Odd Errors: Mechanistic Discovery and Surgical Repair in Transformer Attention
• A Review of Developmental Interpretability in Large Language Models
• CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
• Beyond Technocratic XAI: The Who, What & How in Explanation Design
• BASIC: Boosting Visual Alignment with Intrinsic Refined Embeddings in Multimodal Large Language Models
• Towards Integrated Alignment
• Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations

Related

• Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition
• Transformer Circuit Faithfulness Metrics are not Robust
• InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques
• Circuit Insights: Towards Interpretability Beyond Activations
• Uncovering Intermediate Variables in Transformers using Circuit Probing