Rethinking the Understanding Ability across LLMs through Mutual Information

2025/05/25 by Shaojie Wang, Wang, Shaojie, Sirui Ding +3 · 1 citation
Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Law #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2505.23790

openalex publication_date 2025/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advances in large language models (LLMs) have revolutionized natural language processing, yet evaluating their intrinsic linguistic understanding remains challenging. Moving beyond specialized evaluation tasks, we propose an information-theoretic framework grounded in mutual information (MI) to achieve this. We formalize the understanding as MI between an input sentence and its latent representation (sentence-level MI), measuring how effectively input information is preserved in latent representation. Given that LLMs learn embeddings for individual tokens, we decompose sentence-level MI into token-level MI between tokens and sentence embeddings, establishing theoretical bounds connecting these measures. Based on this foundation, we theoretically derive a computable lower bound for token-level MI using Fano's inequality, which directly relates to token-level recoverability-the ability to predict original tokens from sentence embedding. We implement this recoverability task to comparatively measure MI across different LLMs, revealing that encoder-only models consistently maintain higher information fidelity than their decoder-only counterparts, with the latter exhibiting a distinctive late-layer "forgetting" pattern where mutual information is first enhanced and then discarded. Moreover, fine-tuning to maximize token-level recoverability consistently improves understanding ability of LLMs on tasks without task-specific supervision, demonstrating that mutual information can serve as a foundation for understanding and improving language model capabilities.

Citations

• Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoders
• Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Models
• Layer by Layer: Uncovering Hidden Representations in Language Models
• Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information
• The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance?
• Are Decoder-Only Large Language Models the Silver Bullet for Code Search?
• LLMs' Understanding of Natural Language Revealed
• Multi-Step Reasoning with Large Language Models, a Survey
• Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks
• Your Transformer is Secretly Linear
• LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
• Mixtral of Experts
• Mistral 7B
• Open, Closed, or Small Language Models for Text Classification?
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Using an LLM to Help With Code Understanding
• A Survey on Evaluation of Large Language Models
• A Survey on Evaluation of Large Language Models
• Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study
• Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks
• Testing AI on language comprehension tasks reveals insensitivity to underlying meaning
• MTEB: Massive Text Embedding Benchmark
• OPT: Open Pre-trained Transformer Language Models
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Fact or Fiction: Verifying Scientific Claims
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Pointer Sentinel Mixture Models
• Character-level Convolutional Networks for Text Classification

Cited by

• FlowNIB: An Information Bottleneck Analysis of Bidirectional vs. Unidirectional Language Models

Related

• Diver: Large Language Model Decoding with Span-Level Mutual Information Verification
• The Information of Large Language Model Geometry
• Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs
• LLMs' Understanding of Natural Language Revealed
• Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models