Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method

2023/10/27 by Yukun Zhao, Lingyong Yan, Zhao, Yukun +15 · 6 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Software Engineering Research #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.17918

openalex publication_date 2023/10/27 · openalex created_date 2023/11/01 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have shown great potential in Natural Language Processing (NLP) tasks. However, recent literature reveals that LLMs generate nonfactual responses intermittently, which impedes the LLMs' reliability for further utilization. In this paper, we propose a novel self-detection method to detect which questions that a LLM does not know that are prone to generate nonfactual results. Specifically, we first diversify the textual expressions for a given question and collect the corresponding answers. Then we examine the divergencies between the generated answers to identify the questions that the model may generate falsehoods. All of the above steps can be accomplished by prompting the LLMs themselves without referring to any other external resources. We conduct comprehensive experiments and demonstrate the effectiveness of our method on recently released LLMs, e.g., Vicuna, ChatGPT, and GPT-4.

Cited by

• Hallucinate Less by Thinking More: Aspect-Based Causal Abstention for Large Language Models
• HalluClean: A Unified Framework to Combat Hallucinations in LLMs
• When Facts Change: Probing LLMs on Evolving Knowledge with evolveQA
• Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directions
• Detecting Hallucinations in Authentic LLM-Human Interactions
• Large Language Models Do NOT Really Know What They Don't Know

Related

• Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
• Do Large Language Models Know What They Don't Know?
• Detecting LLM-Generated Short Answers and Effects on Learner Performance
• See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses
• Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation