Conditional Cognitive Biases in LLMs: How Biased User Turns Modulate In-Context Reasoning

2026/05/26 by Sachini Weerasekara, Sagar Kamarthi, Jacqueline Isaacs
Computer Science · #cs.CL #cs.CY #cs.HC #cs.LG

paper · pdf

arxiv created 2026/05/26 · arxiv updated 2026/08/07

Abstract

We present an evaluation of cognitive bias expression in state-of-the-art instruction-tuned LLMs under realistic multi-turn interaction settings. Our work introduces a novel three-condition experimental framework that disentangles the effect of exposure to a biased user turn from the effect of the turn's semantic content, alongside a benchmark of 24,300 jury-validated user prompts spanning all 81 cells of a 9x9 target-human bias interaction matrix. Across eight frontier LLMs, we find that biased conversational context systematically increases bias expression relative to zero-shot baselines in 6 of 8 models. We identify two competing behavioral dynamics underlying this effect: conversational exposure to biased reasoning generally amplifies downstream bias tendencies, while explicitly stated bias cues often trigger alignment-related suppression behaviors that reduce overt bias expression. We release our framework, codebase, and dataset to support future research on context-conditioned cognitive biases and behavioral adaptation in LLMs.

Citations

• A Comprehensive Evaluation of Cognitive Biases in LLMs
• The Llama 3 Herd of Models
• Gemma 2: Improving Open Language Models at a Practical Size
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
• Whose Opinions Do Language Models Reflect?
• A Survey on In-context Learning
• Discovering Language Model Behaviors with Model-Written Evaluations
• Affective Coherence Monitoring for Transformer-Based Language Models
• Out of One, Many: Using Language Models to Simulate Human Samples
• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
• Capturing Failures of Large Language Models via Human Cognitive Biases
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• On the Opportunities and Risks of Foundation Models
• Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
• Calibrate Before Use: Improving Few-Shot Performance of Language Models
• Measuring Massive Multitask Language Understanding
• Language Models are Few-Shot Learners
• Deep reinforcement learning from human preferences
• Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
• Exploring the "planning fallacy": Why people underestimate their task completion times.
• Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias
• The central role of the propensity score in observational studies for causal effects
• The Framing of Decisions and the Psychology of Choice
• Availability: A heuristic for judging frequency and probability
• Opinions and Social Pressure