Large Language Models Encode Clinical Knowledge

2022/12/26 by Karan Singhal, Singhal, Karan, Shekoofeh Azizi +65 · 5 voices · 279 citations
Computer Science · Medicine · #Artificial Intelligence in Healthcare and Education #Text Readability and Simplification #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2212.13138

openalex publication_date 2022/12/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but the quality bar for medical and clinical applications is high. Today, attempts to assess models' clinical knowledge typically rely on automated evaluations on limited benchmarks. There is no standard to evaluate model predictions and reasoning across a breadth of tasks. To address this, we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries; and HealthSearchQA, a new free-response dataset of medical questions searched online. We propose a framework for human evaluation of model answers along multiple axes including factuality, precision, possible harm, and bias. In addition, we evaluate PaLM (a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM, on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA, MedMCQA, PubMedQA, MMLU clinical topics), including 67.6% accuracy on MedQA (US Medical License Exam questions), surpassing prior state-of-the-art by over 17%. However, human evaluation reveals key gaps in Flan-PaLM responses. To resolve this we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, recall of knowledge, and medical reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal important limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLM models for clinical applications.

Cited by

• Instruction-Following Evaluation of Large Vision-Language Models
• MedGemma vs GPT-4: Open-Source and Proprietary Zero-shot Medical Disease Classification from Images
• Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs
• Harnessing Large Language Models for Biomedical Named Entity Recognition
• Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention
• SCTA: An Agentic Framework for Stable and Interpretable Target Gene Discovery from Single-Cell RNA Sequencing
• IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
• Medical world models in healthcare: foundations, applications, and challenges for trustworthy clinical translation
• MyoCardBench: A Real-World Data Benchmark for Evaluating Large Language Models in Clinically Authentic Cardiovascular Care Scenarios
• The Mirage of LLM Guardrails: A Case Study in AI-Assisted Medical Note Manipulation
• Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation
• Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
• Evaluating LLMs as Interpretable Controllers for Dynamical Systems
• Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks
• Socioeconomic Inference in LLM Medical Triage: Same Symptoms, Different ZIP Code
• Structure Over Scale: Schema-Constrained Causal Graphs for RAG
• DeepLens Diagnosis Agent: Agentic Workflow Design Lets a Small Reasoning Model Compete with Frontier LLMs
• Evaluating large language models for diagnostic reasoning from unstructured clinical narratives in epilepsy
• VeriLLMed: Interactive Visual Debugging of Medical Large Language Models with Knowledge Graphs
• Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models
• SpatialBench: Can Agents Analyze Real-World Spatial Biology Data?
• A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning
• Automatic Replication of LLM Mistakes in Medical Conversations
• MediEval: A Unified Medical Benchmark for Patient-Contextual and Knowledge-Grounded Reasoning in LLMs
• Learning Continuous Solvent Effects from Transient Flow Data: A Graph Neural Network Benchmark on Catechol Rearrangement
• Auto-Prompting with Retrieval Guidance for Frame Detection in Logistics
• HARBOR: Holistic Adaptive Risk assessment model for BehaviORal healthcare
• Towards Explainable Conversational AI for Early Diagnosis with Large Language Models
• Exploration of Augmentation Strategies in Multi-modal Retrieval-Augmented Generation for the Biomedical Domain: A Case Study Evaluating Question Answering in Glycobiology
• Mitigating Hallucinations in Healthcare LLMs with Granular Fact-Checking and Domain-Specific Adaptation
• Case Prompting to Mitigate Large Language Model Bias for ICU Mortality Prediction
• SynGP500: A Clinically-Grounded Synthetic Dataset of Australian General Practice Medical Notes
• Do Large Language Models Know What They Don't Know? Kalshibench: A New Benchmark for Evaluating Epistemic Calibration via Prediction Markets
• Georeferencing complex relative locality descriptions with large language models
• Understanding Trust Toward Human versus AI-generated Health Information through Behavioral and Physiological Sensing
• CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcare
• MiniScope: A Least Privilege Framework for Authorizing Tool Calling Agents
• Knowledge-Guided Large Language Model for Automatic Pediatric Dental Record Understanding and Safe Antibiotic Recommendation
• When Medical AI Explanations Help and When They Harm
• A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties
• Chain-of-Image Generation: Toward Monitorable and Controllable Image Generation
• Evolutionary perspective of large language models on shaping research insights into healthcare disparities
• Large Language Models for Education and Research: An Empirical and User Survey-based Analysis
• CLARITY: Medical World Model for Guiding Treatment Decisions by Modeling Context-Aware Disease Trajectories in Latent Space
• Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches
• MCP-AI: Protocol-Driven Intelligence Framework for Autonomous Reasoning in Healthcare
• STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions
• MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis
• Balancing Safety and Helpfulness in Healthcare AI Assistants through Iterative Preference Alignment
• COPE: Chain-Of-Thought Prediction Engine for Open-Source Large Language Model Based Stroke Outcome Prediction from Clinical Notes
• Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92%
• Decentralized Multi-Agent System with Trust-Aware Communication
• UMind-VL: A Generalist Ultrasound Vision-Language Model for Unified Grounded Perception and Comprehensive Interpretation
• Optimizing Life Sciences Agents in Real-Time using Reinforcement Learning
• A Systematic Analysis of Large Language Models with RAG-enabled Dynamic Prompting for Medical Error Detection and Correction
• Medusa: Cross-Modal Transferable Adversarial Attacks on Multimodal Medical Retrieval-Augmented Generation
• Hierarchical Dual-Strategy Unlearning for Biomedical and Healthcare Intelligence Using Imperfect and Privacy-Sensitive Medical Data
• A Counterfactual LLM Framework for Detecting Human Biases: A Case Study of Sex/Gender in Emergency Triage
• Patient-level Information Extraction by Consistent Integration of Textual and Tabular Evidence with Bayesian Networks
• KRAL: Knowledge and Reasoning Augmented Learning for LLM-assisted Clinical Antimicrobial Therapy
• Fairness in Multi-modal Medical Diagnosis with Demonstration Selection
• Distributed Agent Reasoning Across Independent Systems With Strict Data Locality
• Beyond GeneGPT: A Multi-Agent Architecture with Open-Source LLMs for Enhanced Genomic Question Answering
• Transparent Early ICU Mortality Prediction with Clinical Transformer and Per-Case Modality Attribution
• SkinGPT-R1: Adapter-Only Dual Distillation for Efficient Dermatology Reasoning
• MedBench v4: A Robust and Scalable Benchmark for Evaluating Chinese Medical Language Models, Multimodal Models, and Intelligent Agents
• Systematic Reconstruction of Disease Networks from Longitudinal Blood Data for Causal Discovery and Intervention Analysis
• Real-Time Mobile Video Analytics for Pre-arrival Emergency Medical Services
• Can Large Language Models Function as Qualified Pediatricians? A Systematic Evaluation in Real-World Clinical Contexts
• A Novel Hierarchical Integration Method for Efficient Model Merging in Medical LLMs
• Examining the Usage of Generative AI Models in Student Learning Activities for Software Programming
• MedGEN-Bench: Contextually entangled benchmark for open-ended multimodal medical generation
• Human-Level and Beyond: Benchmarking Large Language Models Against Clinical Pharmacists in Prescription Review
• MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications
• Evaluating the Ability of Large Language Models to Identify Adherence to CONSORT Reporting Guidelines in Randomized Controlled Trials: A Methodological Evaluation Study
• MedSumGraph: enhancing GraphRAG for medical QA with summarization and optimized prompts
• Assessing Automated Fact-Checking for Medical LLM Responses with Knowledge Graphs
• Cognitive bias in LLM reasoning compromises interpretation of clinical oncology notes
• 47B Mixture-of-Experts Beats 671B Dense Models on Chinese Medical Examinations
• MedPT: A Massive Medical Question Answering Dataset for Brazilian-Portuguese Speakers
• ClinStructor: AI-Powered Structuring of Unstructured Clinical Texts
• CURENet: Combining Unified Representations for Efficient Chronic Disease Prediction
• Data Poisoning Vulnerabilities Across Healthcare AI Architectures: A Security Threat Analysis
• Mined Prompting and Metadata-Guided Generation for Wound Care Visual Question Answering
• Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning
• How Well Do LLMs Understand Drug Mechanisms? A Knowledge + Reasoning Evaluation Dataset
• CoT-X: An Adaptive Framework for Cross-Model Chain-of-Thought Transfer and Optimization
• Large language models replicate and predict human cooperation across experiments in game theory
• Zero-shot data citation function classification using transformer-based large language models (LLMs)
• Using language models to label clusters of scientific documents
• Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour
• Demo: Statistically Significant Results On Biases and Errors of LLMs Do Not Guarantee Generalizable Results
• Language-Enhanced Generative Modeling for Amyloid PET Synthesis from MRI and Blood Biomarkers
• OceanAI: A Conversational Platform for Accurate, Transparent, Near-Real-Time Oceanographic Insights
• MedRECT: A Medical Reasoning Benchmark for Error Correction in Clinical Texts
• Between Myths and Metaphors: Rethinking LLMs for SRH in Conservative Contexts
• "Koyi Sawaal Nahi Hai": Reimagining Maternal Health Chatbots for Collective, Culturally Grounded Care
• ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models
• EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis
• Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support System
• BioDisclose: An Actionability-Aware Benchmark for Biomedical Safety under Adversarial Elicitation
• Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models
• Multilingual Large Language Models do not comprehend all natural languages to equal degrees
• Artificial intelligence in bioinformatics: a survey
• Generative AI for Healthcare: Fundamentals, Challenges, and Perspectives
• Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
• Towards Transparent Reasoning: What Drives Faithfulness in Large Language Models?
• HACK: Hallucinations Along Certainty and Knowledge Axes
• META-RAG: Meta-Analysis-Inspired Evidence-Re-Ranking Method for Retrieval-Augmented Generation in Evidence-Based Medicine
• PICOs-RAG: PICO-supported Query Rewriting for Retrieval-Augmented Generation in Evidence-Based Medicine
• Label Smoothing Improves Gradient Ascent in LLM Unlearning
• How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation
• User Perceptions of Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
• LM-mixup: Text Data Augmentation via Language Model based Mixup
• KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
• A Multi-faceted Analysis of Cognitive Abilities: Evaluating Prompt Methods with Large Language Models on the CONSORT Checklist
• Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs
• Timely Clinical Diagnosis through Active Test Selection
• Large language models in medicine
• Prompting the Priorities: A First Look at Evaluating LLMs for Vulnerability Triage and Prioritization
• Fine-Tuning MedGemma for Clinical Captioning to Enhance Multimodal RAG over Malaysia CPGs
• From Retrieval to Generation: Unifying External and Parametric Knowledge for Medical Question Answering
• BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks
• Motion2Meaning: A Clinician-Centered Framework for Contestable LLM in Parkinson's Disease Gait Interpretation
• Hearing Health in Home Healthcare: Leveraging LLMs for Illness Scoring and ALMs for Vocal Biomarker Extraction
• Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications
• Disparities in Multilingual LLM-Based Healthcare Q&A
• BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine
• Who's Asking? Simulating Role-Based Questions for Conversational AI Evaluation
• Domain-Contextualized Concept Graphs: A Computable Framework for Knowledge Representation
• TraceCoder: Towards Traceable ICD Coding via Multi-Source Knowledge Integration
• InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training
• Adaptive Minds: Empowering Agents with LoRA-as-Tools
• Circuit Insights: Towards Interpretability Beyond Activations
• CURE: Confidence-driven Unified Reasoning Ensemble Framework for Medical Question Answering
• A survey on retrieval-augmentation generation (RAG) models for healthcare applications
• MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts
• A Survey on Evaluation of Large Language Models
• MedKGEval: A Knowledge Graph-Based Multi-Turn Evaluation Framework for Open-Ended Patient Interactions with Clinical LLMs
• CGBench: Benchmarking Language Model Scientific Reasoning for Clinical Genetics Research
• AgentCaster: Reasoning-Guided Tornado Forecasting
• Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
• Assessing Large Language Models for Structured Medical Order Extraction
• LONGQAEVAL: Designing Reliable Evaluations of Long-Form Clinical QA under Resource Constraints
• ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-test
• Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models
• You only need 4 extra tokens: Synergistic Test-time Adaptation for LLMs
• IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures
• From Explainability to Action: A Generative Operational Framework for Integrating XAI in Clinical Mental Health Screening
• Mephisto: Self-Improving Large Language Model-Based Agents for Automated Interpretation of Multi-band Galaxy Observations
• Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models
• YpathRAG:A Retrieval-Augmented Generation Framework and Benchmark for Pathology
• Resource-Efficient Fine-Tuning of LLaMA-3.2-3B for Medical Chain-of-Thought Reasoning
• CLARITY: Clinical Assistant for Routing, Inference, and Triage
• Evaluation of Clinical Trials Reporting Quality using Large Language Models
• A global log for medical AI
• Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning
• Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines
• Time-To-Inconsistency: A Survival Analysis of Large Language Model Robustness to Adversarial Attacks
• PRISM-Consult: A Panel-of-Experts Architecture for Clinician-Aligned Diagnosis
• Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving
• HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation
• Rethinking Reward Models for Multi-Domain Test-Time Scaling
• Microsaccade-Inspired Probing: Positional Encoding Perturbations Reveal LLM Misbehaviours
• Retrieval-Augmented Framework for LLM-Based Clinical Decision Support
• Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model
• TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
• KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning
• MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning
• Beyond Overall Accuracy: A Psychometric Deep Dive into the Topic-Specific Medical Capabilities of 80 Large Language Models
• MedLA: A Logic-Driven Multi-Agent Framework for Complex Medical Reasoning with Large Language Models
• DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice
• NanoFlux: Adversarial Dual-LLM Evaluation and Distillation For Multi-Domain Reasoning
• RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
• WirelessMathLM: Teaching Mathematical Reasoning for LLMs in Wireless Communications with Reinforcement Learning
• Patient-specific Biomolecular Instruction Tuning
• Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks
• Agentic-AI Healthcare: Multilingual, Privacy-First Framework with MCP Agents
• Position: Human Factors Reshape Adversarial Analysis in Human-AI Decision-Making Systems
• SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips
• RAR²: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval
• When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence
• AgentClinic: a multimodal benchmark for tool-using clinical AI agents
• LLM-assisted systematic review of large language models in clinical medicine
• Artificial intelligence ethics: automation of medical publishing
• Citrus-V: Advancing Medical Foundation Models with Unified Medical Image Grounding for Clinical Reasoning
• Advances in Large Language Models for Medicine
• Exploiting Tree Structure for Credit Assignment in RL Training of LLMs
• AgriDoctor: A Multimodal Intelligent Assistant for Agriculture
• Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness
• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations
• RephQA: Evaluating Readability of Large Language Models in Public Health Question Answering
• Reward Hacking Mitigation using Verifiable Composite Rewards
• Semantic Representation Attack against Aligned Large Language Models
• A Knowledge-driven Adaptive Collaboration of LLMs for Enhancing Medical Decision-making
• Enhancing Retrieval Augmentation via Adversarial Collaboration
• Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learning
• Combining Evidence and Reasoning for Biomedical Fact-Checking
• Performance of Large Language Models in Answering Critical Care Medicine Questions
• A Visualized Framework for Event Cooperation with Generative Agents
• SitLLM: Large Language Models for Sitting Posture Health Understanding via Pressure Sensor Data
• Reinforcement Learning enhanced Online Adaptive Clinical Decision Support via Digital Twin powered Policy and Treatment Effect optimized Reward
• Root Cause Analysis of Radiation Oncology Incidents Using Large Language Models
• Biomedical Hypothesis Explainability with Graph-Based Context Retrieval
• Audited Reasoning Refinement: Fine-Tuning Language Models via LLM-Guided Step-Wise Evaluation and Correction
• Towards Better Health Conversations: The Benefits of Context-seeking
• MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis
• The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models
• A Survey on Retrieval And Structuring Augmented Generation with Large Language Models
• Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
• GrACE: A Generative Approach to Better Confidence Elicitation in Large Language Models
• MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems
• Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications
• Transparency of medical artificial intelligence systems
• HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring
• Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift
• Chatbot To Help Patients Understand Their Health
• Manipulating Transformer-Based Models: Controllability, Steerability, and Robust Interventions
• Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models
• Domain Adaptation for Different Sensor Configurations in 3D Object Detection
• DSRAG: A Domain-Specific Retrieval Framework Based on Document-derived Multimodal Knowledge Graph
• MedQARo: A Large-Scale Benchmark for Medical Question Answering in Romanian
• Machine learning for 2D material–based devices
• RPRO: Ranked Preference Reinforcement Optimization for Enhancing Medical QA and Diagnostic Reasoning
• The Resurgence of GCG Adversarial Attacks on Large Language Models
• A Modality-agnostic Multi-task Foundation Model for Human Brain Imaging
• Automated Clinical Problem Detection from SOAP Notes using a Collaborative Multi-Agent LLM Architecture
• Benchmarking GPT-5 in Radiation Oncology: Measurable Gains, but Persistent Need for Expert Oversight
• HealthProcessAI: A Technical Framework and Proof-of-Concept for LLM-Enhanced Healthcare Process Mining
• OpenAIs HealthBench in Action: Evaluating an LLM-Based Medical Assistant on Realistic Clinical Queries
• End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning
• Embodied AI: Emerging Risks and Opportunities for Policy Action
• Benchmarking GPT-5 for biomedical natural language processing
• Evaluating Language Model Reasoning about Confidential Information
• Survey of Specialized Large Language Model
• Mitigating Clinician Information Overload: Generative AI for Integrated EHR and RPM Data Analysis
• Reasoning LLMs in the Medical Domain: A Literature Survey
• eSkinHealth: A Multimodal Dataset for Neglected Tropical Skin Diseases
• Assessing Pharmacists’ Use and Perception of AI Chatbots in Pharmacy Practice: Cross-Sectional Survey Study
• LLM4Sweat: A Trustworthy Large Language Model for Hyperhidrosis Support
• Organ-Agents: Virtual Human Physiology Simulator via LLMs
• ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicine
• The AI-Fraud Diamond: A Novel Lens for Auditing Algorithmic Deception
• Exploring Efficiency Frontiers of Thinking Budget in Medical Reasoning: Scaling Laws between Computational Resources and Reasoning Quality
• QuarkMed Medical Foundation Model Technical Report
• RPKT: Learning What You Don't -- Know Recursive Prerequisite Knowledge Tracing in Conversational AI Tutors for Personalized Learning
• Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing
• Performance of GPT-5 in Brain Tumor MRI Reasoning
• Reverse Physician-AI Relationship: Full-process Clinical Diagnosis Driven by a Large Language Model
• HiRef: Leveraging Hierarchical Ontology and Network Refinement for Robust Medication Recommendation
• Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
• Can We Trust AI to Govern AI? Benchmarking LLM Performance on Privacy and AI Governance Exams
• Differentiated Information Mining: A Semi-supervised Learning Framework for GNNs
• Evaluating Large Language Models as Expert Annotators
• Capabilities of GPT-5 on Multimodal Medical Reasoning
• A Multi-Agent Approach to Neurological Clinical Reasoning
• EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis
• Towards Experience-Centered AI: A Framework for Integrating Lived Experience in Design and Development
• Synthesizing scientific literature with retrieval-augmented language models
• The Problem of Atypicality in LLM-Powered Psychiatry
• Mediator-Guided Multi-Agent Collaboration among Open-Source Models for Medical Decision-Making
• Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
• Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
• Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models
• Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes
• Benchmarking Foundation Models for Mitotic Figure Classification
• Large Language Model's Multi-Capability Alignment in Biomedical Domain
• Light-IF: Endowing LLMs with Generalizable Reasoning via Preview and Self-Checking for Complex Instruction Following
• Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
• Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modeling
• Large language models accurately identify immunosuppression in intensive care unit patients
• BioDisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluation
• Integrating clinical reasoning into large language model-based diagnosis through etiology-aware attention steering
• A Bayesian Hybrid Parameter-Efficient Fine-Tuning Method for Large Language Models
• CliCARE: Grounding Large Language Models in Clinical Guidelines for Decision Support over Longitudinal Cancer Electronic Health Records
• Predicting Microbial Ontology and Pathogen Risk from Environmental Metadata with Large Language Models
• Leveraging Open-Source Large Language Models for Clinical Information Extraction in Resource-Constrained Settings
• AQUA: A Large Language Model for Aquaculture & Fisheries
• Embeddings to Diagnosis: Latent Fragility under Agentic Perturbations in Clinical LLMs

Discussions

• Large Language Models Encode Clinical Knowledge [hn, 9 points, 1 comments]
• Large Language Models Encode Clinical Knowledge [hn, 2 points, 0 comments]
• Large Language Models Encode Clinical Knowledge [hn, 2 points, 0 comments]
• Large Language Models Encode Clinical Knowledge [hn, 2 points, 0 comments]
• Large Language Models Encode Clinical Knowledge [hn, 1 points, 0 comments]

Related

• Large language models encode clinical knowledge
• Can large language models reason about medical questions?
• Towards Expert-Level Medical Question Answering with Large Language Models
• M-QALM: A Benchmark to Assess Clinical Reading Comprehension and Knowledge Recall in Large Language Models via Question Answering
• Large Language Models in the Clinic: A Comprehensive Benchmark