Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge

2023/06/15 by Zahir Kanjee, Byron Crowe, Adam Rodman · 438 citations
Computer Science · Medicine · #Artificial Intelligence in Healthcare and Education #Artificial intelligence #Clinical Reasoning and Diagnostic Skills #Generative grammar #Machine Learning in Healthcare #Machine learning #Medicine

paper · open access · doi:10.1001/jama.2023.8288

published in JAMA 330(1), 78 (American Medical Association)

openalex publication_date 2023/06/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

This study assesses the diagnostic accuracy of the Generative Pre-trained Transformer 4 (GPT-4) artificial intelligence (AI) model in a series of challenging cases.

Citations

• Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

Cited by

• A global log for medical AI
• Teaching large language models to reason like expert diagnosticians
• Generative AI-Enabled Adaptive Learning Platform: How I Can Help You Pass Your Driving Test?
• Reverse Physician-AI Relationship: Full-process Clinical Diagnosis Driven by a Large Language Model
• Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial
• Leveraging Open-Source Large Language Models for Clinical Information Extraction in Resource-Constrained Settings
• Treatment, evidence, imitation, and chat
• Refine Medical Diagnosis Using Generation Augmented Retrieval and Clinical Practice Guidelines
• Large Language Model Influence on Diagnostic Reasoning
• The impact of generative artificial intelligence on socioeconomic inequalities and policy making
• MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports
• Generative artificial intelligence (GenAI) and decision-making: Legal & ethical hurdles for implementation in mental health
• HealthBench: Evaluating Large Language Models Towards Improved Human Health
• Diagnosis and analysis of plasma phosphorylated tau 217 in a memory clinic cohort: limitations of Generative Pre-trained Transformer-5
• Ensemble Bayesian Inference: Leveraging Small Language Models to Achieve LLM-level Accuracy in Profile Matching Tasks

Related

• GPT4MIA: Utilizing Generative Pre-trained Transformer (GPT-3) as A Plug-and-Play Transductive Model for Medical Image Analysis
• Enhancing Medical Task Performance in GPT-4V: A Comprehensive Study on Prompt Engineering Strategies
• Evaluating Use of Generative Artificial Intelligence in Clinical Pathology Practice: Opportunities and the Way Forward
• STAGER checklist: Standardized Testing and Assessment Guidelines for Evaluating Generative AI Reliability
• Generative Artificial Intelligence Considerations Beyond Natural Language Processing: Computer Vision Applications in Health Care