Heimersheim, Stefan
- Open Problems in Mechanistic Interpretability
2025/01/27 by Lee Sharkey, Bilal Chughtai, Sharkey, Lee +58 · 7 voices · 35 citations
Computer Science · #Natural Language Processing Techniques #Statistical and Computational Modeling #cs.LG
- Towards Automated Circuit Discovery for Mechanistic Interpretability
2023/04/28 by Arthur Conmy, Augustine N. Mavor-Parker, Conmy, Arthur +7 · 76 citations
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG)
- How to use and interpret activation patching
2024/04/23 by Stefan Heimersheim, Heimersheim, Stefan, Neel Nanda +1 · 35 citations
Computer Science · Business, Management and Accounting · #Usability and User Interface Design #Business Process Modeling and Analysis #Intelligent Tutoring Systems and Adaptive Learning
- Detecting Strategic Deception Using Linear Probes
2025/02/05 by Nicholas Goldowsky-Dill, Bilal Chughtai, Goldowsky-Dill, Nicholas +5 · 19 citations
Computer Science · Psychology · #Advanced Malware Detection Techniques #Deception detection and forensic psychology #FOS: Computer and information sciences #Machine Learning (cs.LG) #Network Security and Intrusion Detection
- Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
2025/01/24 by Dan Braun, Lucius Bushnaq, Braun, Dan +7 · 8 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques
- Evolution of SAE Features Across Layers in LLMs
2024/10/11 by Balcells, Daniel, Lerner, Benjamin, Oesterle, Michael +2 · 5 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG)
- Joint analysis constraints on the physics of the first galaxies with low frequency radio astronomy data
2023/01/09 by Bevins, Harry T. J., Heimersheim, Stefan, Abril-Cabezas, Irene +5 · 2 citations
#Cosmology and Nongalactic Astrophysics (astro-ph.CO) #FOS: Physical sciences #Instrumentation and Methods for Astrophysics (astro-ph.IM)
- You can remove GPT2's LayerNorm by fine-tuning
2024/09/06 by Stefan Heimersheim, Heimersheim, Stefan · 3 citations
Computer Science · #Advanced Data Storage Technologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques
- Rapid and Late Cosmic Reionization Driven by Massive Galaxies: a Joint Analysis of Constraints from 21-cm, Lyman Line & CMB Data Sets
2025/04/13 by Peter Sims, H. T. J. Bevins, Sims, Peter H. +15 · 3 citations
Physics and Astronomy · #Astronomy and Astrophysical Research #Cosmology and Nongalactic Astrophysics (astro-ph.CO) #FOS: Physical sciences #Instrumentation and Methods for Astrophysics (astro-ph.IM) #Radio Astronomy Observations and Technology
- Benchmarking Deception Probes via Black-to-White Performance Boosts
2025/07/16 by Avi Parrack, Parrack, Avi, Carlo Leonardo Attubato +3 · 4 citations
Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Network Security and Intrusion Detection
- Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
2025/07/03 by Baroni, Luca, Khara, Galvin, Schaeffer, Joachim +2 · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG)
- The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
2024/05/17 by Lucius Bushnaq, Bushnaq, Lucius, Stefan Heimersheim +17 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications
- Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
2024/05/17 by Lucius Bushnaq, Bushnaq, Lucius, Jake Mendel +13 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Statistical and Computational Modeling