Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantage

2025/10/03 by Ángela López-Cardona, Sebastián Idesis, López-Cardona, Ángela +7 · 1 citation
Neuroscience · Psychology · #Neurobiology of Language and Bilingualism #Action Observation and Synchronization #Embodied and Extended Cognition

paper · pdf · doi:10.48550/arxiv.2510.17833

Abstract

Do brains and language models converge toward the same internal representations of the world? Recent years have seen a rise in studies of neural activations and model alignment. In this work, we review 25 fMRI-based studies published between 2023 and 2025 and explicitly confront their findings with two key hypotheses: (i) the Platonic Representation Hypothesis -- that as models scale and improve, they converge to a representation of the real world, and (ii) the Intermediate-Layer Advantage -- that intermediate (mid-depth) layers often encode richer, more generalizable features. Our findings provide converging evidence that models and brains may share abstract representational structures, supporting both hypotheses and motivating further research on brain-model alignment.

Citations

• Language models align with brain regions that represent concepts across modalities
• Brains and language models converge on a shared conceptual space across different languages
• Brain-tuned Speech Models Better Reflect Speech Processing Stages in the Brain
• Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
• Multi-modal brain encoding models for multi-modal stimuli
• Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
• Vectors from Larger Language Models Predict Human Reading Time and fMRI Data More Poorly when Dimensionality Expansion is Controlled
• Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions
• Layer by Layer: Uncovering Hidden Representations in Language Models
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Inducing Human-like Biases in Moral Reasoning Language Models
• Improving Semantic Understanding in Speech Language Models via Brain-tuning
• Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models
• LLaVA-OneVision: Easy Visual Task Transfer
• The Llama 3 Herd of Models
• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• From Word Embedding to Reading Embedding Using Large Language Model, EEG and Eye-tracking
• Scalable Pre-training of Large Autoregressive Image Models
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Instruction-tuning Aligns LLMs to the Human Brain
• Speech language models lack important brain-relevant semantics
• Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language
• Baichuan 2: Open Large-scale Language Models
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
• Mapping Brains with Language Models: A Survey
• Brain encoding models based on multimodal transformers can transfer across language and vision
• Scaling laws for language encoding models in fMRI
• InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
• ImageBind: One Embedding Space To Bind Them All
• mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
• Visual Instruction Tuning
• DINOv2: Learning Robust Visual Features without Supervision
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
• DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies
• The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes
• Training language models to summarize narratives improves brain alignment
• Joint processing of linguistic properties in brains and language models
• Robust Speech Recognition via Large-Scale Weak Supervision
• Language models and brains align due to more than next-word prediction and word-level information
• Scaling Instruction-Finetuned Language Models
• MTEB: Massive Text Embedding Benchmark
• TVLT: Textless Vision-Language Transformer
• BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning
• GIT: A Generative Image-to-text Transformer for Vision and Language
• OPT: Open Pre-trained Transformer Language Models
• VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence
• FLAVA: A Foundational Language And Vision Alignment Model
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
• BookSum: A Collection of Datasets for Long-form Narrative Summarization
• ViViT: A Video Vision Transformer
• Learning Transferable Visual Models From Natural Language Supervision
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Measuring Massive Multitask Language Understanding
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• Longformer: The Long-Document Transformer
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language\n Generation, Translation, and Comprehension
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)
• Attention Is All You Need
• Deep Residual Learning for Image Recognition
• Distributed Representations of Words and Phrases and their Compositionality
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Representing Beauty: Towards a Participatory but Objective Latent Aesthetics

Related

• Joint processing of linguistic properties in brains and language models
• Language models and brains align due to more than next-word prediction and word-level information
• Mapping Brains with Language Models: A Survey
• Does injecting linguistic structure into language models lead to better alignment with brain recordings?
• Language models align with brain regions that represent concepts across modalities