Layers at Similar Depths Generate Similar Activations Across LLM Architectures

2025/04/03 by Christopher Wolfram, Aaron Schein, Wolfram, Christopher +1 · 1 voice · 6 citations
Computer Science · Engineering · #VLSI and Analog Circuit Testing #Advancements in Photolithography Techniques #Advancements in Semiconductor Devices and Circuit Design

paper · pdf · doi:10.48550/arxiv.2504.08775

Abstract

How do the latent spaces used by independently-trained LLMs relate to one another? We study the nearest neighbor relationships induced by activations at different layers of 24 open-weight LLMs, and find that they 1) tend to vary from layer to layer within a model, and 2) are approximately shared between corresponding layers of different models. Claim 2 shows that these nearest neighbor relationships are not arbitrary, as they are shared across models, but Claim 1 shows that they are not "obvious" either, as there is no single set of nearest neighbor relationships that is universally shared. Together, these suggest that LLMs generate a progression of activation geometries from layer to layer, but that this entire progression is largely shared between models, stretched and squeezed to fit into different architectures.

Citations

• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• Attention Is All You Need

Cited by

• Correlated Errors in Large Language Models
• Harnessing the Universal Geometry of Embeddings
• Mapping Overlaps in Benchmarks through Perplexity in the Wild
• Large Language Models Show Signs of Alignment with Human Neurocognition During Abstract Reasoning
• Progressive Depth Up-scaling via Optimal Transport
• Multiple Streams of Knowledge Retrieval: Enriching and Recalling in Transformers

Discussions

• Know what’s wild? Layers at Similar Depths Generate Similar Activations Across LLM Architectures Different optimizers. Different number of layers. Different hyperparameters. Different data. Strangely [bsky, 21 points, 3 comments]

Related

• Secret mixtures of experts inside your LLM
• Uncovering Hierarchical Structure in LLM Embeddings with δ-Hyperbolicity, Ultrametricity, and Neighbor Joining
• The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology
• Model Stitching: Looking For Functional Similarity Between Representations
• Unraveling the Localized Latents: Learning Stratified Manifold Structures in LLM Embedding Space with Sparse Mixture-of-Experts