Detoxifying Large Language Models via Knowledge Editing

2024/03/21 by Mengru Wang, Wang, Mengru, Ningyu Zhang +17 · 10 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Machine Learning (cs.LG) #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2403.14472

openalex publication_date 2024/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper investigates using knowledge editing techniques to detoxify Large Language Models (LLMs). We construct a benchmark, SafeEdit, which covers nine unsafe categories with various powerful attack prompts and equips comprehensive metrics for systematic evaluation. We conduct experiments with several knowledge editing approaches, indicating that knowledge editing has the potential to detoxify LLMs with a limited impact on general performance efficiently. Then, we propose a simple yet effective baseline, dubbed Detoxifying with Intraoperative Neural Monitoring (DINM), to diminish the toxicity of LLMs within a few tuning steps via only one instance. We further provide an in-depth analysis of the internal mechanism for various detoxifying approaches, demonstrating that previous methods like SFT and DPO may merely suppress the activations of toxic parameters, while DINM mitigates the toxicity of the toxic parameters to a certain extent, making permanent adjustments. We hope that these insights could shed light on future work of developing detoxifying approaches and the underlying knowledge mechanisms of LLMs. Code and benchmark are available at https://github.com/zjunlp/EasyEdit.

Cited by

• EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
• An Empirical Study of Sample Selection Strategies for Large Language Model Repair
• SoK: Taxonomy and Evaluation of Prompt Security in Large Language Models
• SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models
• DSCD: Large Language Model Detoxification with Self-Constrained Decoding
• Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models
• Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions
• Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
• A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functions
• A Survey on Training-free Alignment of Large Language Models

Related

• Precision Knowledge Editing: Enhancing Safety in Large Language Models
• Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing
• Detoxification of Large Language Models through Output-layer Fusion with a Calibration Model
• SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxification
• DSCD: Large Language Model Detoxification with Self-Constrained Decoding