Performance and Practical Considerations of Large and Small Language Models in Clinical Decision Support in Rheumatology

2025/07/10 by Felde, Sabine, Buchkremer, Rüdiger, Chehab, Gamal +4
#Artificial Intelligence (cs.AI) #C.4 #Computation and Language (cs.CL) #FOS: Computer and information sciences #H.3.3 #H02.403.720.750 #I.2.7 #I.2.9 #J.3 #L01.224.050.375 #L01.224.900.500 (Primary) #L01.700.508.300 #N04.452.758.625 (Secondary) #N04.590

paper · doi:10.48550/arxiv.2507.07983

Abstract

Large language models (LLMs) show promise for supporting clinical decision-making in complex fields such as rheumatology. Our evaluation shows that smaller language models (SLMs), combined with retrieval-augmented generation (RAG), achieve higher diagnostic and therapeutic performance than larger models, while requiring substantially less energy and enabling cost-efficient, local deployment. These features are attractive for resource-limited healthcare. However, expert oversight remains essential, as no model consistently reached specialist-level accuracy in rheumatology.

Citations

• Enhancing Large Language Models with Retrieval-Augmented Generation: A Radiology-Specific Approach
• A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
• Retrieval-Augmented Generation for Generative Artificial Intelligence in Medicine
• Design and Evaluation of a Retrieval-Augmented Generation Architecture for OWASP Security Data
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Mixture-of-Experts with Expert Choice Routing
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Related

• Performance of Large Language Models in Supporting Medical Diagnosis and Treatment
• Evaluating large language models in medical applications: a survey
• Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyond
• Large Language Models as Agents in the Clinic
• Large Language Models for Medicine: A Survey