Can an LLM Learn Preferences from Choice Data?

2024/01/14 by Jeongbin Kim, Kim, Jeongbin, Matthew Kovach +7 · 2 citations
Decision Sciences · Engineering · #Energy Load and Power Forecasting #FOS: Economics and business #Forecasting Techniques and Applications #General Economics (econ.GN) #Stock Market Forecasting Methods

paper · pdf · doi:10.48550/arxiv.2401.07345

openalex publication_date 2024/01/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Can large language models (LLMs) learn a decision maker's preferences from observed choices and generate preference-consistent recommendations in new situations? We propose a portable Simulate-Recommend-Evaluate framework that tests preference learning from revealed-choice data by comparing LLM recommendations with optimal choices implied by known preference primitives. We apply the framework to choice under uncertainty using the disappointment aversion model. Recommendation accuracy improves as models observe more choices, but learning is heterogeneous across preference types and LLMs: GPT learns risk aversion better than disappointment aversion, Gemini performs best in high disappointment-aversion regions, and Claude shows the broadest effective learning across parameter regions.

Cited by

• The Innate Economic Preferences of Language Models
• Adverse reactions to the use of large language models in social interactions

Related

• Evaluating and Aligning Human Economic Risk Preferences in LLMs
• Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context
• AI as Decision-Maker: Ethics and Risk Preferences of LLMs
• LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theory
• The Emergence of Economic Rationality of GPT