Taming Object Hallucinations with Verified Atomic Confidence Estimation

2025/11/12 by Liu, Jiarui, Xuan, Weihao, Jin, Zhijing +1 · 2 citations
#Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2511.09228

Abstract

Multimodal Large Language Models (MLLMs) often suffer from hallucinations, particularly errors in object existence, attributes, or relations, which undermine their reliability. We introduce TACO (Verified Atomic Confidence Estimation), a simple framework that mitigates hallucinations through self-verification and confidence calibration without relying on external vision experts. TACO decomposes responses into atomic queries, paraphrases them to reduce sensitivity to wording, and estimates confidence using self-consistency (black-box) or self-confidence (gray-box) aggregation, before refining answers with a language model. Experiments on five benchmarks (POPE, MME, HallusionBench, AMBER, and MM-Hal Bench) with two MLLMs (LLaVA-1.5-7B and CogVLM2) show that TACO consistently outperforms direct prompting and Visual Contrastive Decoding, reduces systematic biases, and improves confidence calibration, demonstrating its effectiveness in enhancing the faithfulness of MLLMs.

Citations

• Promptception: How Sensitive Are Large Multimodal Models to Prompts?
• Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification
• Large Language Model Confidence Estimation via Black-Box Access
• Linguistic Calibration of Long-Form Generations
• Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
• Calibrating Large Language Models with Sample Consistency
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
• OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
• AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation
• Aligning Large Multimodal Models with Factually Augmented RLHF
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
• Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement
• Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
• Large Language Models are Better Reasoners with Self-Verification
• Teaching Models to Express Their Uncertainty in Words
• Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation
• QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
• A Survey on Automated Fact-Checking
• Revisiting the Calibration of Modern Neural Networks
• Reducing conversational agents' overconfidence through linguistic calibration
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
• On the Inference Calibration of Neural Machine Translation
• Calibration of Pre-trained Transformers
• Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning
• Reasoning Over Semantic-Level Graph for Fact Checking
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• On Calibration of Modern Neural Networks
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• TRUE: Re-evaluating Factual Consistency Evaluation

Cited by

• Witness Evidence Portfolios: Single-Prefill Risk Detection for Closed Multimodal Answers
• Re-Centering Humans in LLM Personalization

Related

• LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations
• Atomic Calibration of LLMs in Long-Form Generations
• Measuring Epistemic Humility in Multimodal Large Language Models
• Chain-of-Verification Reduces Hallucination in Large Language Models
• Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models