Steering Awareness: Detecting Activation Steering from Within

2025/11/26 by Africa, David Demitri, Rivera, Joshua Fonseca · 1 voice · 2 citations
Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #Component (thermodynamics) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Focus (optics) #Generalization #Introspection #Memorization #Metacognition #Multimodal Machine Learning Applications #Neurobiology of Language and Bilingualism #Representation (politics) #Security token #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2511.21399

published in ArXiv.org

openalex publication_date 2025/11/26 · openalex created_date 2025/11/28 · openalex updated_date 2026/08/05

Abstract

Activation steering -- adding a vector to a model's residual stream to modify its behavior -- is widely used in safety evaluations as if the model cannot detect the intervention. We test this assumption, introducing steering awareness: a model's ability to infer, during its own forward pass, that a steering vector was injected and what concept it encodes. After fine-tuning, seven instruction-tuned models develop strong steering awareness on held-out concepts; the best reaches 95.5% detection, 71.2% concept identification, and zero false positives on clean inputs. This generalizes to unseen steering vector construction methods when their directions have high cosine similarity to the training distribution but not otherwise, indicating a geometric detector rather than a generic anomaly detector. Surprisingly, detection does not confer resistance; on both factual and safety benchmarks, detection-trained models are consistently more susceptible to steering than their base counterparts. Mechanistically, steering awareness arises not from a localized circuit, but from a distributed transformation that progressively rotates diverse injected vectors into a shared detection direction. Activation steering should therefore not be considered an invisible intervention in safety evaluations.

Citations

• Latent Introspection: Models Can Detect Prior Concept Injections
• Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
• Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors
• Depth-Wise Activation Steering for Honest Language Models
• Steering Evaluation-Aware Language Models to Act Like They Are Deployed
• Privileged Self-Access Matters for Introspection in AI
• Persona Vectors: Monitoring and Controlling Character Traits in Language Models
• Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
• Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Models
• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
• LatentQA: Teaching LLMs to Decode Activations Into Natural Language
• Looking Inward: Language Models Can Learn About Themselves by Introspection
• The Llama 3 Herd of Models
• Gemma 2: Improving Open Language Models at a Practical Size
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
• Steering Llama 2 via Contrastive Activation Addition
• LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
• Representation Engineering: A Top-Down Approach to AI Transparency
• Steering Language Models With Activation Engineering
• The Hydra Effect: Emergent Self-repair in Language Model Computations
• Code of "Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs"
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
• Locating and Editing Factual Associations in GPT
• LoRA: Low-Rank Adaptation of Large Language Models
• Probing Classifiers: Promises, Shortcomings, and Advances
• Understanding intermediate layers using linear classifier probes

Cited by

• Detecting the Disturbance: A Nuanced View of Introspective Abilities in LLMs
• Can LLMs Introspect? A Reality Check

Discussions

• So you can definitely train a model to detect (and report on!) activation steering. So they can explain what concepts are being injected and identify (most of the time) when it is happening. arxiv.org [bsky, 2 points, 1 comments]

Related

• Emergent Introspective Awareness in Large Language Models
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect
• Looking Inward: Language Models Can Learn About Themselves by Introspection
• Mechanisms of Introspective Awareness
• Emergent Introspection in AI is Content-Agnostic