Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data

2025/08/01 by Saad Ali Imran, Rob Lamb, Imran, Sohaib +3
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2508.00741

openalex publication_date 2025/08/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) are trained on large corpora, yet it is unclear whether they can reason about the information present within their training data. We design experiments to study out-of-context abduction in LLMs, the ability to infer the most plausible explanations for observations using relevant facts present in training data. We train treatment LLMs on names and behavior descriptions of fictitious chatbots, but not on examples of dialogue with the chatbots. We find that OpenAI's GPT 4o LLM can correctly infer at least one chatbot's name after observing example responses characteristic of that chatbot. We also find that previously training GPT 4o on descriptions of a chatbot's behavior allows it to display behaviors more characteristic of the chatbot when iteratively trained to display such behaviors. Our results have implications for situational awareness in LLMs and, therefore, for AI safety.

Citations

• A sketch of an AI control safety case
• Tell me about yourself: LLMs are aware of their learned behaviors
• Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
• Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
• AI models collapse when trained on recursively generated data
• Large Language Models are Limited in Out-of-Context Knowledge Reasoning
• Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
• Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
• Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?
• UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations
• Scheming AIs: Will AIs fake alignment during training in order to get power?
• Implicit meta-learning may lead language models to trust more reliable sources
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Taken out of context: On measuring situational awareness in LLMs
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
• Kronfluence: Influence Functions with Eigenvalue-corrected Kronecker-Factored Approximate Curvature
• Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
• Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning
• Role-Play with Large Language Models
• Role play with large language models
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• Towards Reasoning in Large Language Models: A Survey
• Emergent Analogical Reasoning in Large Language Models
• The Alignment Problem from a Deep Learning Perspective
• Scaling Laws for Neural Language Models
• Abductive Commonsense Reasoning
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

Related

• Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
• Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
• Taken out of context: On measuring situational awareness in LLMs
• Can We Infer Confidential Properties of Training Data from LLMs?
• Can large language models explore in-context?