Humans and LLMs Diverge on Probabilistic Inferences

2026/02/26 by Gaurav Kamath, Sreenath Madathil, Sebastian Schuster +2 · 1 voice
Computer Science · #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2602.23546

Abstract

Human reasoning often involves working over limited information to arrive at probabilistic conclusions. In its simplest form, this involves making an inference that is not strictly entailed by a premise, but rather only likely given the premise. While reasoning LLMs have demonstrated strong performance on logical and mathematical tasks, their behavior on such open-ended, non-deterministic inferences remains largely unexplored. We introduce ProbCOPA, a dataset of 210 handcrafted probabilistic inferences in English, each annotated for inference likelihood by 25--30 human participants. We find that human responses are graded and varied, revealing probabilistic judgments of the inferences in our dataset. Comparing these judgments with responses from eight state-of-the-art reasoning LLMs, we show that models consistently fail to produce human-like distributions. Finally, analyzing LLM reasoning chains, we find evidence of a common reasoning pattern used to evaluate such inferences. Our findings reveal persistent differences between humans and LLMs, and underscore the need to evaluate reasoning beyond deterministic settings.

Citations

• Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
• Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
• Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
• Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing
• Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
• Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
• Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!
• Let's Think Var-by-Var: Large Language Models Enable Ad Hoc Probabilistic Reasoning
• Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models
• What Are the Odds? Language Models Are Capable of Probabilistic Reasoning
• Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?
• Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
• Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models
• Calibrating Large Language Models Using Their Generations Only
• VariErr NLI: Separating Annotation Error from Human Label Variation
• Ecologically Valid Explanations for Label Variation in NLI
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
• We're Afraid Language Models Aren't Modeling Ambiguity
• Whose Opinions Do Language Models Reflect?
• Teaching Models to Express Their Uncertainty in Words
• What Can We Learn from Collective Human Opinions on Natural Language Inference Data?
• Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
• Abductive Commonsense Reasoning
• A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
• Ordinal Common-sense Inference
• A large annotated corpus for learning natural language inference
• A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001–2010
• On belief bias in syllogistic reasoning.
• Reasoning about a Rule

Discussions

• ..LLMs Diverge on Probabilistic Inferences]: https://arxiv.org/abs/2602.23546 (8/8) [bsky, 0 points, 0 comments]

Related

• Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
• How reliable are LLMs when it comes to playing dice?
• What Are the Odds? Language Models Are Capable of Probabilistic Reasoning
• Tracing the ongoing emergence of human-like reasoning in Large Language Models
• Benchmarking LLM Competence on Logical Inference over Probability Operators