LLM Safety From Within: Detecting Harmful Content with Internal Representations

2026/04/20 by Difan Jiao, Yilun Liu, Ye Yuan +4 · 1 voice
Computer Science · #Adversarial Robustness in Machine Learning #Generalization #Generative grammar #Guard (computer science) #Hate Speech and Cyberbullying Detection #Inference #Siren (mythology) #Topic Modeling #cs.AI

paper · pdf · open access · doi:10.48550/arxiv.2604.18519

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2026/04/20 · arxiv published 2026/04/20 · arxiv updated 2026/04/20 · openalex created_date 2026/04/22 · openalex updated_date 2026/07/28

Abstract

Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.

Citations

• Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
• Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?
• Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
• Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment
• Generative or Discriminative? Revisiting Text Classification in the Era of Transformers
• PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
• Layer by Layer: Uncovering Hidden Representations in Language Models
• Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails
• Lightweight Safety Classification Using Pruned Language Models
• HSF: Defending against Jailbreak Attacks with Hidden State Filtering
• Safety Layers in Aligned Large Language Models: The Key to LLM Security
• ShieldGemma: Generative AI Content Moderation Based on Gemma
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
• AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts
• Do Llamas Work in English? On the Latent Language of Multilingual Transformers
• SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification
• Linear Representations of Sentiment in Large Language Models
• The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
• Linearity of Relation Decoding in Transformer Language Models
• Code of "Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs"
• BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
• Finding Neurons in a Haystack: Case Studies with Sparse Probing
• Eliciting Latent Predictions from Transformers with the Tuned Lens
• A Holistic Approach to Undesired Content Detection in the Real World
• HateBERT: Retraining BERT for Abusive Language Detection in English
• Scaling Laws for Neural Language Models
• How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Optuna: A Next-generation Hyperparameter Optimization Framework
• Deep Layer Aggregation
• Understanding intermediate layers using linear classifier probes
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Discussions

• Papers In The Park is at New Stadium today. Because of rain. We’re covering: arxiv.org/abs/2604.18519 [bsky, 0 points, 0 comments]

Related

• Trust The Typical
• From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring
• SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals
• Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences
• BingoGuard: LLM Content Moderation Tools with Risk Levels