The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models

2025/02/03 by Zhiyuan Xu, Xu, Zhiyuan, Joseph Gardiner +3 · 5 citations
Materials Science · #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning in Materials Science #Nanocluster Synthesis and Applications

paper · pdf · doi:10.48550/arxiv.2502.01225

openalex publication_date 2025/02/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models are typically trained on vast amounts of data during the pre-training phase, which may include some potentially harmful information. Fine-tuning attacks can exploit this by prompting the model to reveal such behaviours, leading to the generation of harmful content. In this paper, we focus on investigating the performance of the Chain of Thought based reasoning model, DeepSeek, when subjected to fine-tuning attacks. Specifically, we explore how fine-tuning manipulates the model's output, exacerbating the harmfulness of its responses while examining the interaction between the Chain of Thought reasoning and adversarial inputs. Through this study, we aim to shed light on the vulnerability of Chain of Thought enabled models to fine-tuning attacks and the implications for their safety and ethical deployment.

Cited by

• Steering in the Shadows: Causal Amplification for Activation Space Attacks in Large Language Models
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails
• AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models
• ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection
• UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases

Related

• BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models
• Thought Purity: A Defense Framework For Chain-of-Thought Attack
• AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models
• Where Do Reasoning Models Refuse?
• Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models