How Effective Is Constitutional AI in Small LLMs? A Study on DeepSeek-R1 and Its Peers

2025/02/01 by Menke, Antonio-Gabriel Chacón, Tan, Phan Xuan · 1 citation
#Artificial Intelligence (cs.AI) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2503.17365

Abstract

Recent incidents highlight safety risks in Large Language Models (LLMs), motivating research into alignment methods like Constitutional AI (CAI). This paper explores CAI's self-critique mechanism on small, uncensored 7-9B parameter models: DeepSeek-R1-8B, Gemma-2-9B, Llama 3.1-8B, and Qwen2.5-7B. We show that while Llama-based models exhibited significant harm reduction through self-critique, other architectures demonstrated less improvement in harm detection after abliteration. These results suggest CAI's effectiveness may vary depending on model architecture and reasoning capabilities.

Cited by

• Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots

Related

• Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B
• IterAlign: Iterative Constitutional Alignment of Large Language Models
• Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applications
• Affective Coherence Monitoring for Transformer-Based Language Models
• Responsible Federated LLMs via Safety Filtering and Constitutional AI