Do Large Language Models Reason Causally Like Us? Even Better?

2025/02/14 by Hanna M. Dettki, Dettki, Hanna M., Brenden M. Lake +5 · 3 citations
Computer Science · #Topic Modeling #Explainable Artificial Intelligence (XAI) #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2502.10215

Abstract

Causal reasoning is a core component of intelligence. Large language models (LLMs) have shown impressive capabilities in generating human-like text, raising questions about whether their responses reflect true understanding or statistical patterns. We compared causal reasoning in humans and four LLMs using tasks based on collider graphs, rating the likelihood of a query variable occurring given evidence from other variables. LLMs' causal inferences ranged from often nonsensical (GPT-3.5) to human-like to often more normatively aligned than those of humans (GPT-4o, Gemini-Pro, and Claude). Computational model fitting showed that one reason for GPT-4o, Gemini-Pro, and Claude's superior performance is they didn't exhibit the "associative bias" that plagues human causal reasoning. Nevertheless, even these LLMs did not fully capture subtler reasoning patterns associated with collider graphs, such as "explaining away".

Cited by

• If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models
• Beyond computational equivalence: the behavioral inference principle for machine consciousness
• Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem

Related

• Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
• Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?
• Causality for Large Language Models
• C²P: Featuring Large Language Models with Causal Reasoning
• Is Knowledge All Large Language Models Needed for Causal Reasoning?