2025/09/18 by Anthony Howell, Jieshu Wang, Howell, Anthony +7 · 4 voices · 1 citation
Computer Science · Decision Sciences · Social Sciences · #Academic Publishing and Open Access #Audit #Computational and Text Analysis Methods #Face (sociological concept) #Identity (music) #Peer review #Prestige #Set (abstract data type) #cs.CY #scientometrics and bibliometrics research
paper · pdf · doi:10.48550/arxiv.2509.15122
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Large language models (LLMs) are playing an increasingly integral, though largely informal, role in scholarly peer review. Yet it remains unclear whether LLMs reproduce the biases observed in human decision-making. We adapt a resume-style audit to scientific publishing, developing a multi-role LLM simulation (editor/reviewer) that evaluates a representative set of high-quality manuscripts across the physical, biological, and social sciences under randomized author identities (institutional prestige, gender, race). The audit reveals a strong and consistent institutional-prestige bias: identical papers attributed to low-prestige affiliations face a significantly higher risk of rejection, despite only modest differences in LLM-assessed quality. To probe mechanisms, we generate synthetic CVs for the same author profiles; these encode large prestige-linked disparities and an inverted prestige-tenure gradient relative to national benchmarks. The results suggest that both domain norms and prestige-linked priors embedded in training data shape paper-level outcomes once identity is visible, converting affiliation into a decisive status cue.