Diagnosis Uncertain Models For Medical Risk Prediction

2023/06/29 by Alexander Peysakhovich, Rich Caruana, Peysakhovich, Alexander +3
Computer Science · Economics, Econometrics and Finance · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Health Systems, Economic Evaluations, Quality of Life #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2306.17337

openalex publication_date 2023/06/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider a patient risk models which has access to patient features such as vital signs, lab values, and prior history but does not have access to a patient's diagnosis. For example, this occurs in a model deployed at intake time for triage purposes. We show that such `all-cause' risk models have good generalization across diagnoses but have a predictable failure mode. When the same lab/vital/history profiles can result from diagnoses with different risk profiles (e.g. E.coli vs. MRSA) the risk estimate is a probability weighted average of these two profiles. This leads to an under-estimation of risk for rare but highly risky diagnoses. We propose a fix for this problem by explicitly modeling the uncertainty in risk prediction coming from uncertainty in patient diagnoses. This gives practitioners an interpretable way to understand patient risk beyond a single risk number.

Related

• Uncertainty of risk estimates from clinical prediction models: rationale, challenges, and approaches
• The fundamental problem of risk prediction for individuals: health AI, uncertainty, and personalized medicine
• Uncertainty estimation for classification and risk prediction on medical tabular data
• Domain constraints improve risk prediction when outcome data is missing
• Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications