Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight

2025/12/22 by Junze Ye, Daniel Tawfik, Ye, Junze +9
Computer Science · Medicine · #Applications (stat.AP) #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #FOS: Computer and information sciences #Machine Learning and Data Classification #Machine Learning in Healthcare

paper · doi:10.48550/arxiv.2512.19691

openalex publication_date 2025/12/22 · openalex created_date 2025/12/24 · openalex updated_date 2026/07/28

Abstract

Reference labels for machine-learning benchmarks are increasingly synthesized with LLM assistance, but their reliability remains underexamined. We audit MedCalc-Bench, a clinical benchmark for medical score computation whose labels were partly derived with LLM assistance, and develop a scalable physician-in-the-loop stewardship pipeline to reassess them. At least 27% of test labels are likely erroneous or incomputable. On a 50-instance subset validated by physicians, our recomputed labels agree with physician ground truth 74% of the time (95% CI, 60-84%) versus 20% for the originals (95% CI, 11-33%). Using original labels to evaluate frontier LLMs underestimates accuracy by 16-23 percentage points. In a controlled reinforcement-learning experiment, a model trained on recomputed labels outperforms one trained on originals by 13.5 percentage points (95% CI, 10.6-16.6%) on physician-labeled instances, and this advantage extends to related medical tasks. LLM-assisted benchmarks can propagate systematic errors into both evaluation and post-training unless actively stewarded.

Citations

• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations
• AI-based Clinical Decision Support for Primary Care: A Real-World Study
• Rethinking Test-Time Scaling for Medical AI: Model and Task-Aware Strategies for LLMs and VLMs
• Med-U1: Incentivizing Unified Medical Reasoning in LLMs via Large-scale Reinforcement Learning
• Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning
• MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
• Qwen3 Technical Report
• Understanding R1-Zero-Like Training: A Critical Perspective
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power
• Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance
• A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?
• Training Language Models to Self-Correct via Reinforcement Learning
• AI models collapse when trained on recursively generated data
• MedCalc-Bench: Evaluating Large Language Models for Medical Calculations
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Training language models to follow instructions with human feedback
• Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
• PRAG: Paninian Retrieval-Augmented Generation for Safety-Critical Medical Question Answering
• Estimating the Attributable Cost of Physician Burnout in the United States
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study

Related

• Automatic Construction of Clinical Scoring Systems with LLM Agents
• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations
• Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
• Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
• Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise