2025/12/02 by Davis, Bee Rosa · 1 voice
Engineering · Neuroscience · #Ferroelectric and Negative Capacitance Devices #Functional Brain Connectivity Studies #Neurobiology of Language and Bilingualism
paper · doi:10.5281/zenodo.17783723
openalex publication_date 2025/12/02 · openalex created_date 2025/12/03 · openalex updated_date 2026/07/01
This paper introduces spectral geometry methods for analyzing the internal organization of transformer language models. By treating attention patterns as weighted graphs and computing their Laplacian spectra, we extract geometric invariants—eigenvalue gaps, heat kernel signatures, and multi-scale curvature estimates—that reveal functional structure invisible to standard interpretability techniques. Key Findings: Experiments on the GPT-2 model family (12–36 layers, 124M–774M parameters) and TinyLlama (1.1B parameters) demonstrate: Universal U-shaped curvature profile: High curvature at entry/exit layers, minimal curvature in middle layers—consistent across all models tested Task-specific activation patterns: Factual recall peaks at ~50% depth with high spectral gaps; pattern matching concentrates in early layers (<20% depth) Spectral gap collapse predicts failure: Observable geometric distress signal before incorrect outputs are generated Conserved two-stage memory circuit: Early activation triggering (~6% depth) followed by stable transmission through high-spectral-gap regions (~64% depth) Geometric effort as retrieval signal: Distinguishes grounded factual recall from confabulation independent of output correctness Hallucination Detection Breakthrough The most significant finding: forced correct and forced incorrect completions show identical geometric effort, while confabulation prompts show markedly lower effort. This indicates that geometric signatures measure whether the model is attempting grounded retrieval, not whether the retrieved answer is correct—enabling inference-time detection of confabulation without external verification. Practical Applications: Heat-kernel pruning achieves 14× better robustness than variance-based pruning at 55% head retention Spectral gap monitoring enables real-time failure prediction Geometry-based routing for mixture-of-experts architectures Targeted fine-tuning via geometric objectives This work establishes spectral geometry as a principled framework for transformer interpretability and demonstrates that geometric properties of attention patterns encode meaningful information about model computation. Keywords transformer interpretability; spectral graph theory; heat kernels; attention mechanisms; geometric deep learning; hallucination detection; language models; neural network analysis; Laplacian spectrum; curvature Related Publications: Davis, B. R. (2025). The Davis Conjecture on Semantic Coherence: Context Windows as Holonomy Horizons in Functorial Transformers. Zenodo. https://doi.org/10.5281/zenodo.17780665 Davis, B. R. (2025). The Field Equations of Semantic Coherence: A Geometric Theory of Meaning, Curvature, and Reasoning in Transformer Architectures. Zenodo. https://doi.org/10.5281/zenodo.17771796 Davis, B. R. (2025). The Geometry of Generative Reasoning: Gauge-Theoretic Transformers as Realizations of Semantic Sameness. Zenodo. https://doi.org/10.5281/zenodo.17718659 Davis, B. R. (2025). The Geometry of Sameness: An ε-Equivalence of Translation and Distance. Zenodo. https://doi.org/10.5281/zenodo.17642422 Davis, B. R. (2025). The Davis Manifold: Geometry-First Detection with Compositional Error Budgets. Zenodo. https://doi.org/10.5281/zenodo.17642038 Related Books: The Geometry of Sameness: Riemannian Equivalence of Translation and Distance for Semantic Detection (Geometric Intelligence) https://a.co/d/iJTrdrO Hidden Variable: Unlocking Patterns in a World Obsessed with Structure (Geometric Intelligence) https://a.co/d/fUCn0AV License Creative Commons Attribution 4.0 International Resource Type Publication / Preprint Version 1.0.0