Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation

2025/05/29 by Hongxiang Zhang, Hao Chen, Zhang, Hongxiang +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2505.23657

openalex publication_date 2025/05/29 · openalex created_date 2025/10/12 · openalex updated_date 2026/07/28

Abstract

Recent decoding methods improve the factuality of large language models (LLMs) by refining how the next token is selected during generation. These methods typically operate at the token level, leveraging internal representations to suppress superficial patterns. Nevertheless, LLMs remain prone to hallucinations, especially over longer contexts. In this paper, we propose Active Layer-Contrastive Decoding (ActLCD), a novel decoding strategy that actively decides when to apply contrasting layers during generation. By casting decoding as a sequential decision-making problem, ActLCD employs a reinforcement learning policy guided by a reward-aware classifier to optimize factuality beyond the token level. Our experiments demonstrate that ActLCD surpasses state-of-the-art methods across five benchmarks, showcasing its effectiveness in mitigating hallucinations in diverse generation scenarios.

Cited by

• Evaluating Program Semantics Reasoning with Type Inference in System F

Related

• Lower Layers Matter: Alleviating Hallucination via Multi-Layer Fusion Contrastive Decoding with Truthfulness Refocused
• Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs
• Mitigating Hallucination in Multimodal LLMs with Layer Contrastive Decoding
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations
• Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models