Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values

2025/05/30 by John P. Dickerson, Hadi Hosseini, Dickerson, John P. +5 · 2 voices
Computer Science · #Artificial Intelligence (cs.AI) #Computers and Society (cs.CY) #FOS: Computer and information sciences #I.2.1 #I.2.11 #I.2.7 #Machine Learning (cs.LG) #cs.AI #cs.CY #cs.LG

paper · pdf · doi:10.48550/arxiv.2506.00079

arxiv published 2025/05/30 · arxiv updated 2026/04/19

Abstract

The rapid integration of Large Language Models (LLMs) in high-stakes decision-making -- such as allocating scarce resources like donor organs -- raises critical questions about their alignment with human moral values. We systematically evaluate the behavior of several prominent LLMs against human preferences in kidney allocation scenarios and show that LLMs: i) exhibit stark deviations from human values in prioritizing various attributes, and ii) in contrast to humans, LLMs rarely express indecision, opting for deterministic decisions even when alternative indecision mechanisms (e.g., coin flipping) are provided. Nonetheless, we show that low-rank supervised fine-tuning with few samples is often effective in improving both decision consistency and calibrating indecision modeling. These findings illustrate the necessity of explicit alignment strategies for LLMs in moral/ethical domains.

Citations

• Generative Social Choice: The Next Generation
• The Convergent Ethics of AI? Analyzing Moral Foundation Priorities in Large Language Models with a Multi-Framework Approach
• Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions
• Personality-Driven Decision-Making in LLM-Based Autonomous Agents
• Susceptibility of Large Language Models to User-Driven Factors in Medical Queries
• Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy
• None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering
• Can AI Model the Complexities of Human Moral Decision-making? A Qualitative Study of Kidney Allocation Decisions
• Psychological Counseling Ability of Large Language Models
• Normative Evaluation of Large Language Models with Everyday Moral Dilemmas
• Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?
• Evaluating Binary Decision Biases in Large Language Models: Implications for Fair Agent-Based Financial Simulations
• Right vs. Right: Can LLMs Make Tough Choices?
• The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
• Moral Alignment for LLM Agents
• ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
• On The Stability of Moral Preferences: A Problem with Computational Elicitation Methods
• Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models
• Are Large Language Models Consistent over Value-laden Questions?
• Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context
• MoralBench: Moral Evaluation of LLMs
• Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
• MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
• Forcing Diffuse Distributions out of Language Models
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
• Generative Social Choice
• Exploring the psychology of LLMs' Moral and Legal Reasoning
• Evaluating the Moral Beliefs Encoded in LLMs
• GPT-4 Technical Report
• Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
• Fine-tuning language models to find agreement among humans with diverse preferences
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
• Indecision Modeling
• Fair Allocation of Scarce Medical Resources in the Time of Covid-19
• Fairness and Abstraction in Sociotechnical Systems
• LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
• Can Machines Learn Morality? The Delphi Experiment
• THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED

Discussions

• Now in the Morals & Norms session at #FAccT2026 Starting out w/ "Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values" by Dickerson et. al Paper Link: arxiv.org/abs/2506.00079 #FAccT2 [bsky, 3 points, 0 comments]
• I really am of two minds about this type of research: https://www.arxiv.org/abs/2506.00079 On the one hand, probing the output of an LLM and comparing against thoughts of (selected) people shows us *s [bsky, 0 points, 1 comments]

Related

• Aligning with Heterogeneous Preferences for Kidney Exchange
• Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback
• Indecision Modeling
• The Moral Mind(s) of Large Language Models
• Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment