H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs
2025/12/01 by Cheng Gao, Huimin Chen, Gao, Cheng +9 · 14 voices
Medicine · Computer Science · Engineering · #Schizophrenia research and treatment #Adversarial Robustness in Machine Learning #Ferroelectric and Negative Capacitance Devices
paper · pdf · doi:10.48550/arxiv.2512.01797
Abstract
Large language models (LLMs) frequently generate hallucinations -- plausible but factually incorrect outputs -- undermining their reliability. While prior work has examined hallucinations from macroscopic perspectives such as training data and objectives, the underlying neuron-level mechanisms remain largely unexplored. In this paper, we conduct a systematic investigation into hallucination-associated neurons (H-Neurons) in LLMs from three perspectives: identification, behavioral impact, and origins. Regarding their identification, we demonstrate that a remarkably sparse subset of neurons (less than 0.1% of total neurons) can reliably predict hallucination occurrences, with strong generalization across diverse scenarios. In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors. Concerning their origins, we trace these neurons back to the pre-trained base models and find that these neurons remain predictive for hallucination detection, indicating they emerge during pre-training. Our findings bridge macroscopic behavioral patterns with microscopic neural mechanisms, offering insights for developing more reliable LLMs.
Cited by
Discussions
- Fantastic mechinterp paper showing that you can identify specific neurons which cause hallucinated answers in LLMs, and that these neurons are specifically associated with the language model trying to [bsky, 170 points, 6 comments]
- On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs [hn, 30 points, 44 comments]
- china patched lying btw. theyre gonna upstream it to humans in the next couple weeks [bsky, 17 points, 1 comments]
- H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs [lobsters, 11 points, 0 comments]
- Create people pleasing tendencies? They are awful Please please PLEASE have some compassion arxiv.org/abs/2512.01797 [bsky, 9 points, 0 comments]
- On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs https:// arxiv.org/abs/2512.01797 # arxiv # llm # llms [mastodon, 1 points, 0 comments]
- Og Paper: arxiv.org/abs/2512.01797 [bsky, 0 points, 0 comments]
- Tsinghua identified the neurons that cause AI hallucination. They survive alignment unchanged. The fix has to be architectural. [lemmy, 0 points, 0 comments]
- Origin of Hallucination in LLMs, The physical source of hallucinations has found https:// arxiv.org/abs/2512.01797 # HackerNews # OriginOfHallucination # LLMs # AIResearch # MachineLearning # NeuralNe [mastodon, 0 points, 0 comments]
- A root cause of confabulation has been determined, but there's no way to fix it at this time. I imagine that they'll probably end up running several models concurrently at various 'temperatures' and t [bsky, 0 points, 0 comments]
- AI hallucinations solved? 🤯 Tsinghua researchers found 'H-neurons' that control AI's honesty! Adjusting these can make AI more reliable. Learn how: arxiv.org/pdf/2512.01797 #AI #Hallucinations [bsky, 0 points, 0 comments]
- Origin of Hallucination in LLMs, The physical source of hallucinations has found #HackerNews https://arxiv.org/abs/2512.01797 [bsky, 0 points, 0 comments]
- H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs https://lobste.rs/s/5ymlmt #ai [bsky, 0 points, 0 comments]
- Origin of Hallucination in LLMs, The physical source of hallucinations has found https://arxiv.org/abs/2512.01797 (https://news.ycombinator.com/item?id=46354773) [bsky, 0 points, 0 comments]
Related