Street-Level AI: Are Large Language Models Ready for Real-World Judgments?

2025/08/11 by Gaurab Pokharel, Pokharel, Gaurab, Shafkat Farabi +5 · 1 voice · 2 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computers and Society (cs.CY) #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #cs.AI #cs.CY

paper · pdf · doi:10.48550/arxiv.2508.08193

openalex publication_date 2025/08/11 · arxiv published 2025/08/11 · arxiv updated 2025/09/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A surge of recent work explores the ethical and societal implications of large-scale AI models that make "moral" judgments. Much of this literature focuses either on alignment with human judgments through various thought experiments or on the group fairness implications of AI judgments. However, the most immediate and likely use of AI is to help or fully replace the so-called street-level bureaucrats, the individuals deciding to allocate scarce social resources or approve benefits. There is a rich history underlying how principles of local justice determine how society decides on prioritization mechanisms in such domains. In this paper, we examine how well LLM judgments align with human judgments, as well as with socially and politically determined vulnerability scoring systems currently used in the domain of homelessness resource allocation. Crucially, we use real data on those needing services (maintaining strict confidentiality by only using local large models) to perform our analyses. We find that LLM prioritizations are extremely inconsistent in several ways: internally on different runs, between different LLMs, and between LLMs and the vulnerability scoring systems. At the same time, LLMs demonstrate qualitative consistency with lay human judgments in pairwise testing. Findings call into question the readiness of current generation AI systems for naive integration in high-stakes societal decision-making.

Citations

• Exploring Persona-dependent LLM Alignment for the Moral Machine Experiment
• Evaluating Large Language Models for Fair and Reliable Organ Allocation
• Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Discretionary Trees: Understanding Street-Level Bureaucracy via Machine Learning
• Large language models in healthcare and medical domain: A review
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
• LLaMA: Open and Efficient Foundation Language Models
• Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support
• Rank Centrality: Ranking from Pair-wise Comparisons
• Sample Size Requirements for Estimating Pearson, Kendall and Spearman Correlations
• Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach
• RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS

Cited by

• Exploring question answering: metric analysis and evaluation framework for enhanced interpretability
• MolMark: Safeguarding Molecular Structures through Learnable Atom-Level Watermarking

Discussions

• ..robustness, and contextual grounding before entrusting machines with moral judgments that profoundly impact vulnerable lives. Read more: https://arxiv.org/abs/2508.08193v1 (10/10) [bsky, 0 points, 0 comments]

Related

• Local Justice and the Algorithmic Allocation of Societal Resources
• Evaluating and Improving Value Judgments in AI: A Scenario-Based Study on Large Language Models' Depiction of Social Conventions
• The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
• Toward AI Matching Policies in Homeless Services: A Qualitative Study with Policymakers
• The social consequences of AI delegation