Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions

2025/05/16 by Jian-Qiao Zhu, Hanbo Xie, Zhu, Jian-Qiao +7 · 1 voice · 3 citations
Computer Science · Medicine · #Explainable Artificial Intelligence (XAI) #Multimodal Machine Learning Applications #Artificial Intelligence in Healthcare and Education

paper · pdf · doi:10.48550/arxiv.2505.11614

Abstract

A central goal of cognitive modeling is to develop models that not only predict human behavior but also provide insight into the underlying cognitive mechanisms. While neural network models trained on large-scale behavioral data often achieve strong predictive performance, they typically fall short in offering interpretable explanations of the cognitive processes they capture. In this work, we explore the potential of pretrained large language models (LLMs) to serve as dual-purpose cognitive models--capable of both accurate prediction and interpretable explanation in natural language. Specifically, we employ reinforcement learning with outcome-based rewards to guide LLMs toward generating explicit reasoning traces for explaining human risky choices. Our findings demonstrate that this approach produces high-quality explanations alongside strong quantitative predictions of human decisions.

Citations

• Reinforcement Learning from Human Feedback
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• Tell me about yourself: LLMs are aware of their learned behaviors
• Centaur: a foundation model of human cognition
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
• Bayes in the age of intelligent machines
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
• Language Models are Few-Shot Learners
• Heuristic Decision Making

Cited by

• Addressing Longstanding Challenges in Cognitive Science with Language Models
• This human study did not involve human subjects: Validating LLM simulations as behavioral evidence
• Finetuning LLMs for Human Behavior Prediction in Social Science Experiments

Discussions

• この研究では、人間の意思決定を説明する能力を持つ大規模言語モデルのトレーニング方法を提案しています。従来のニューラルネットワークモデルは、人間の行動を予測することはできるものの、その背後にある認知プロセスを解釈することが難しいという問題がありました。そこで、この研究では、大規模言語モデルに強化学習を用いて、人間の危険な選択を説明する明示的な推論手順を生成させました。結果として、高品質の解釈とともに [bsky, 0 points, 0 comments]

Related

• Predicting and Understanding Human Action Decisions: Insights from Large Language Models and Cognitive Instance-Based Learning
• Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice
• Turning large language models into cognitive models
• Large Language Models are Biased Reinforcement Learners
• On the Modeling Capabilities of Large Language Models for Sequential Decision Making