Representation Learning on a Random Lattice

2025/04/28 by A. Brill, Brill, Aryeh
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Disordered Systems and Neural Networks (cond-mat.dis-nn) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #FOS: Physical sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2504.20197

openalex publication_date 2025/04/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Decomposing a deep neural network's learned representations into interpretable features could greatly enhance its safety and reliability. To better understand features, we adopt a geometric perspective, viewing them as a learned coordinate system for mapping an embedded data distribution. We motivate a model of a generic data distribution as a random lattice and analyze its properties using percolation theory. Learned features are categorized into context, component, and surface features. The model is qualitatively consistent with recent findings in mechanistic interpretability and suggests directions for future research.

Citations

• Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry
• Sparse Autoencoders Do Not Find Canonical Units of Analysis
• Neural Scaling Laws Rooted in the Data Distribution
• Decomposing The Dark Matter of Sparse Autoencoders
• The Geometry of Concepts: Sparse Autoencoder Feature Structure
• A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
• Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
• Scaling and evaluating sparse autoencoders
• The Platonic Representation Hypothesis
• Mechanistic Interpretability for AI Safety -- A Review
• The Linear Representation Hypothesis and the Geometry of Large Language Models
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks
• Finding Neurons in a Haystack: Case Studies with Sparse Probing
• A Note on Normalized Emergence Timing (in Pythia Language Model Evaluations)
• The Quantization Model of Neural Scaling
• Progress measures for grokking via mechanistic interpretability
• Governance Architecture for Neural Network Superposition: A Structural Solution to Hallucination via Routing and Interference Filtering
• Towards Understanding Grokking: An Effective Theory of Representation Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Adversarial Examples Are Not Bugs, They Are Features
• Embedded Agency
• Neural Discrete Representation Learning
• Representation Learning: A Review and New Perspectives
• Representation Learning: A Review and New Perspectives
• Cayley Trees and Bethe Lattices, a concise analysis for mathematicians and physicists
• Family resemblances: Studies in the internal structure of categories
• Gradient-based learning applied to document recognition

Related

• Representation Learning: A Review and New Perspectives
• Representation Learning: A Review and New Perspectives
• Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry
• When Representations Align: Universality in Representation Learning Dynamics
• An unsupervised tour through the hidden pathways of deep neural networks