On Subjective Uncertainty Quantification and Calibration in Natural Language Generation

2024/06/07 by Ziyu Wang, Chris Holmes, Wang, Ziyu +1 · 1 voice · 3 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2406.05213

openalex publication_date 2024/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Applications of large language models often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This is due to the need to identify task-specific uncertainties (e.g., about the semantics) which appears difficult to define in general cases. This work addresses these challenges from a perspective of Bayesian decision theory, starting from the assumption that our utility is characterized by a similarity measure that compares a generated response with a hypothetical true response. We discuss how this assumption enables principled quantification of the model's subjective uncertainty and its calibration. We further derive a measure for epistemic uncertainty, based on a missing data perspective and its characterization as an excess risk. The proposed methods can be applied to black-box language models. We illustrate the methods on question answering and machine translation tasks. Our experiments provide a principled evaluation of task-specific calibration, and demonstrate that epistemic uncertainty offers a promising deferral strategy for efficient data acquisition in in-context learning.

Cited by

• Calibrating Semantic Uncertainty from Observable Language-Model Probabilities
• SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?
• Identification and Learning of Semantic Observation Kernels: Partial Observation, Uniform Recovery, & Minimax Limits

Discussions

• This is a simple and good paper, which somehow nobody working on these things cites, or even seems to be aware of arxiv.org/abs/2406.05213 It is simple idea that seems useful; it formulates the subjec [bsky, 27 points, 2 comments]

Related

• Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuning
• ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models
• Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning
• Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey
• Finetuning Language Models to Emit Linguistic Expressions of Uncertainty