Emotion Concepts and their Function in a Large Language Model

2026/04/09 by Nicholas Sofroniew, Isaac Kauvar, William Saunders +13 · 1 voice · 5 citations
Computer Science · Social Sciences · #Computational and Text Analysis Methods #Context (archaeology) #ENCODE #Expression (computer science) #Function (biology) #Human language #Key (lock) #Language and cultural evolution #Phenomenon #Relevance (law) #Topic Modeling #cs.AI #cs.CL

paper · pdf · open access · doi:10.48550/arxiv.2604.07729

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2026/04/09 · arxiv published 2026/04/09 · arxiv updated 2026/04/09 · openalex created_date 2026/04/11 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) sometimes appear to exhibit emotional reactions. We investigate why this is the case in Claude Sonnet 4.5 and explore implications for alignment-relevant behavior. We find internal representations of emotion concepts, which encode the broad concept of a particular emotion and generalize across contexts and behaviors it might be linked to. These representations track the operative emotion concept at a given token position in a conversation, activating in accordance with that emotion's relevance to processing the present context and predicting upcoming text. Our key finding is that these representations causally influence the LLM's outputs, including Claude's preferences and its rate of exhibiting misaligned behaviors such as reward hacking, blackmail, and sycophancy. We refer to this phenomenon as the LLM exhibiting functional emotions: patterns of expression and behavior modeled after humans under the influence of an emotion, which are mediated by underlying abstract representations of emotion concepts. Functional emotions may work quite differently from human emotions, and do not imply that LLMs have any subjective experience of emotions, but appear to be important for understanding the model's behavior.

Cited by

• Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control
• Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests
• Do Large Language Models Have Emotions?
• Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment
• Consciousness as Uncommon Self-Knowledge: A Synergistic Information Framework
• Functional Emotions or Situational Contexts? A Discriminating Test from the Mythos Preview System Card

Discussions

• Anthropic team study on Claude Sonnet 4.5 reveals LLMs simulate emotions, influencing behavior and alignment, crucial for ethical AI development. arxiv.org/abs/2604.07729 [bsky, 2 points, 0 comments]

Related

• Do Large Language Models Have Emotions?
• Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models
• Emergence of Hierarchical Emotion Organization in Large Language Models
• Do LLMs "Feel"? Emotion Circuits Discovery and Control
• Mechanistic Interpretability of Emotion Inference in Large Language Models