FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning

2025/10/05 by Xu Shen, Shen, Xu, Song Wang +11 · 2 citations
Social Sciences · #Artificial Intelligence (cs.AI) #Evolutionary Game Theory and Cooperation #FOS: Computer and information sciences #Religion, Spirituality, and Psychology

paper · pdf · doi:10.48550/arxiv.2510.04040

openalex publication_date 2025/10/05 · openalex created_date 2025/10/09 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) increasingly rely on Chain-of-Thought (CoT) prompting to improve problem-solving and provide seemingly transparent explanations. However, growing evidence shows that CoT often fail to faithfully represent the underlying reasoning process, raising concerns about their reliability in high-risk applications. Although prior studies have focused on mechanism-level analyses showing that CoTs can be unfaithful, they leave open the practical challenge of deciding whether a specific trajectory is faithful to the internal reasoning of the model. To address this gap, we introduce FaithCoT-Bench, a unified benchmark for instance-level CoT unfaithfulness detection. Our framework establishes a rigorous task formulation that formulates unfaithfulness detection as a discriminative decision problem, and provides FINE-CoT (Faithfulness instance evaluation for Chain-of-Thought), an expert-annotated collection of over 1,000 trajectories generated by four representative LLMs across four domains, including more than 300 unfaithful instances with fine-grained causes and step-level evidence. We further conduct a systematic evaluation of eleven representative detection methods spanning counterfactual, logit-based, and LLM-as-judge paradigms, deriving empirical insights that clarify the strengths and weaknesses of existing approaches and reveal the increased challenges of detection in knowledge-intensive domains and with more advanced models. To the best of our knowledge, FaithCoT-Bench establishes the first comprehensive benchmark for instance-level CoT faithfulness, setting a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.

Citations

• Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction
• Evaluating Large Language Models for Cross-Lingual Retrieval
• Towards Cognitively-Faithful Decision-Making Models to Improve AI Alignment
• BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks
• Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
• Agentar-DeepFinance-100K: A Large-Scale Financial Dataset via Systematic Chain-of-Thought Synthesis Optimization
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
• How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?
• Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systems
• LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
• Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models
• AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning
• Reasoning Models Don't Always Say What They Think
• A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
• Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
• In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
• On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective
• Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps
• Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models
• Multi-Agent Simulator Drives Language Models for Legal Intensive Interaction
• Demystifying Long Chain-of-Thought Reasoning in LLMs
• Detecting Strategic Deception Using Linear Probes
• Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?
• Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
• On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep
• Dissociation of Faithful and Unfaithful Reasoning in LLMs
• Chain of Thoughtlessness? An Analysis of CoT in Planning
• The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models
• Large Language Models for Data Annotation and Synthesis: A Survey
• Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning
• Explainability for Large Language Models: A Survey
• Graph of Thoughts: Solving Elaborate Problems with Large Language Models
• Measuring Faithfulness in Chain-of-Thought Reasoning
• Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
• Large Language Models as Commonsense Knowledge for Large-Scale Task Planning
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
• Self-planning Code Generation with Large Language Models
• Self-Planning Code Generation with Large Language Models
• MathPrompter: Mathematical Reasoning using Large Language Models
• Faithful Chain-of-Thought Reasoning
• Towards Reasoning in Large Language Models: A Survey
• Large Language Models are Zero-Shot Reasoners
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Towards Faithfully Interpretable NLP Systems: How should we define and\n evaluate faithfulness?
• Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems

Cited by

• QuiLL: An LLM-Based Vulnerability Assessment Framework for the Wild
• Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models

Related

• Measuring Faithfulness in Chain-of-Thought Reasoning
• Two Regimes of Chain-of-Thought Unfaithfulness: Behavioral Detection Fails Where Models Are Wrong
• FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
• Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity
• Chain-of-Thought Unfaithfulness as Disguised Accuracy