Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders

2025/05/20 by Agam Goyal, V. P. Gladis Pushpa Rathi, Goyal, Agam +9 · 1 citation
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2505.14536

openalex publication_date 2025/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) are now ubiquitous in user-facing applications, yet they still generate undesirable toxic outputs, including profanity, vulgarity, and derogatory remarks. Although numerous detoxification methods exist, most apply broad, surface-level fixes and can therefore easily be circumvented by jailbreak attacks. In this paper we leverage sparse autoencoders (SAEs) to identify toxicity-related directions in the residual stream of models and perform targeted activation steering using the corresponding decoder vectors. We introduce three tiers of steering aggressiveness and evaluate them on GPT-2 Small and Gemma-2-2B, revealing trade-offs between toxicity reduction and language fluency. At stronger steering strengths, these causal interventions surpass competitive baselines in reducing toxicity by up to 20%, though fluency can degrade noticeably on GPT-2 Small depending on the aggressiveness. Crucially, standard NLP benchmark scores upon steering remain stable, indicating that the model's knowledge and general abilities are preserved. We further show that feature-splitting in wider SAEs hampers safety interventions, underscoring the importance of disentangled feature learning. Our findings highlight both the promise and the current limitations of SAE-based causal interventions for LLM detoxification, further suggesting practical guidelines for safer language-model deployment.

Citations

• MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance
• LLM Social Simulations Are a Promising Research Method
• Training large language models on narrow tasks can lead to broad misalignment
• Compromising Honesty and Harmlessness in Language Models via Deception Attacks
• Sparse Autoencoders Do Not Find Canonical Units of Analysis
• Position: Editing Large Language Models Poses Serious Safety Risks
• AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
• LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
• SLM-Mod: Small Language Models Surpass LLMs at Content Moderation
• An Adversarial Perspective on Machine Unlearning for AI Safety
• A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders
• Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
• Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders
• Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval
• A Complete Survey on LLM-based AI Chatbots
• Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity
• Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
• A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
• Simulating Opinion Dynamics with Networks of LLM-based Agents
• Self-Detoxifying Language Models via Toxification Reversal
• LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Emergent Linear Representations in World Models of Self-Supervised Sequence Models
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Word Embeddings Are Steers for Language Models
• Toxicity in ChatGPT: Analyzing Persona-assigned Language Models
• Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
• Desiderata for Representation Learning: A Causal Perspective
• DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
• Detoxifying Language Models Risks Marginalizing Minority Voices
• RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
• GeDi: Generative Discriminator Guided Sequence Generation
• Language Models are Few-Shot Learners
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• Plug and Play Language Models: A Simple Approach to Controlled Text Generation
• Fine-Tuning Language Models from Human Preferences
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• The Woman Worked as a Babysitter: On Biases in Language Generation
• Energy and Policy Considerations for Deep Learning in NLP
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Proximal Policy Optimization Algorithms
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
• The Measurement of Observer Agreement for Categorical Data

Cited by

• ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Concepts

Related

• Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing
• Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing
• Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking
• Large Language Models can be Strong Self-Detoxifiers
• Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM