The Geometry of Categorical and Hierarchical Concepts in Large Language Models

2024/06/03 by Kiho Park, Yo Joong Choe, Park, Kiho +5 · 7 voices · 24 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2406.01506

openalex publication_date 2024/06/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The linear representation hypothesis is the informal idea that semantic concepts are encoded as linear directions in the representation spaces of large language models (LLMs). Previous work has shown how to make this notion precise for representing binary concepts that have natural contrasts (e.g., male, female) as directions in representation space. However, many natural concepts do not have natural contrasts (e.g., whether the output is about an animal). In this work, we show how to extend the formalization of the linear representation hypothesis to represent features (e.g., is_animal) as vectors. This allows us to immediately formalize the representation of categorical concepts as polytopes in the representation space. Further, we use the formalization to prove a relationship between the hierarchical structure of concepts and the geometry of their representations. We validate these theoretical results on the Gemma and LLaMA-3 large language models, estimating representations for 900+ hierarchically related concepts using data from WordNet.

Cited by

• Context Is King: How In-Context Specification Shapes the Geometry of Concepts
• Emergent Hierarchical Monosemantic Neurons from the Group-Contrastive Forward-Forward Algorithm
• When Models Manipulate Manifolds: The Geometry of a Counting Task
• Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
• Semantic Tuning of Single Neurons in the Human Medial Temporal Lobe
• From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
• From superposition to sparse codes: interpretable representations in neural networks
• Shared Global and Local Geometry of Language Model Embeddings
• The Lattice Representation Hypothesis of Large Language Models
• Rethinking Intrinsic Dimension Estimation in Neural Representations
• Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks
• Steering grids for sparse-autoencoder features: when a top-context label names an activation regime rather than a causal axis
• Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality
• On the Theoretical Foundation of Sparse Dictionary Learning in Mechanistic Interpretability
• Better World Models Can Lead to Better Post-Training Performance
• Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models
• The Geometry of Reasoning: Flowing Logics in Representation Space
• Steering Embedding Models with Geometric Rotation: Mapping Semantic Relationships Across Languages and Models
• Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling
• Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
• Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
• The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models
• Native Logical and Hierarchical Representations with Subspace Embeddings
• Semantic Structure in Large Language Model Embeddings

Discussions

• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [hn, 123 points, 15 comments]
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [hn, 7 points, 0 comments]
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [lobsters, 4 points, 2 comments]
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [hn, 3 points, 0 comments]
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models [hn, 3 points, 0 comments]
• arxiv.org/abs/2406.01506 [bsky, 0 points, 0 comments]
• The Geometry of Categorical and Hierarchical Concepts in Large Language Models (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]

Related

• The Linear Representation Hypothesis and the Geometry of Large Language Models
• The Lattice Representation Hypothesis of Large Language Models
• On the Origins of Linear Representations in Large Language Models
• The Origins of Representation Manifolds in Large Language Models
• Semantic Structure in Large Language Model Embeddings