Context-dependent outcome encoding in human reinforcement learning

2021/07/20 by Stefano Palminteri, Maël Lebreton · 1 voice
Decision Sciences · Social Sciences · Neuroscience · #Decision-Making and Behavioral Economics #Experimental Behavioral Economics Studies #Neural and Behavioral Psychology Studies

paper · doi:10.1016/j.cobeha.2021.06.006

openalex publication_date 2021/07/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A wealth of evidence in perceptual and economic decision-making research suggests that the subjective assessment of one option is influenced by the context. A series of studies provides evidence that the same coding principles apply to situations where decisions are shaped by past outcomes, that is, in reinforcement-learning situations. In bandit tasks, human behavior is explained by models assuming that individuals do not learn the objective value of an outcome, but rather its subjective, context-dependent representation. We argue that, while such outcome context-dependence may be informationally or ecologically optimal, it concomitantly undermines the capacity to generalize value-based knowledge to new contexts — sometimes creating apparent decision paradoxes.

Citations

• Decision by sampling
• Habits without values.
• Prospect Theory: An Analysis of Decision under Risk

Discussions

• In decision-making, context matters a lot. Previous work shows that humans don't value rewards in absolute terms — instead, we compare them to a reference point. But do other animals do the same? We [bsky, 1 points, 1 comments]

Related

• Implications of Human Irrationality for Reinforcement Learning
• Foundations of Reinforcement Learning and Interactive Decision Making
• A Smoothed Analysis of the Greedy Algorithm for the Linear Contextual Bandit Problem
• Learning Optimal Behavior Through Reasoning and Experiences
• Elucidating attentional mechanisms underlying value normalization in human reinforcement learning