2023/04/07 by Yilin Ning, Victor Volovici, Ning, Yilin +7
Computer Science · Health Professions · Medicine · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computers and Society (cs.CY) #FOS: Computer and information sciences #Healthcare cost, quality, practices #Machine Learning (cs.LG) #Machine Learning in Healthcare
paper · pdf · doi:10.48550/arxiv.2304.03779
openalex publication_date 2023/04/07 · openalex created_date 2023/04/12 · openalex updated_date 2026/07/28
A prediction model is most useful if it generalizes beyond the development data with external validations, but to what extent should it generalize remains unclear. In practice, prediction models are externally validated using data from very different settings, including populations from other health systems or countries, with predictably poor results. This may not be a fair reflection of the performance of the model which was designed for a specific target population or setting, and may be stretching the expected model generalizability. To address this, we suggest to externally validate a model using new data from the target population to ensure clear implications of validation performance on model reliability, whereas model generalizability to broader settings should be carefully investigated during model development instead of explored post-hoc. Based on this perspective, we propose a roadmap that facilitates the development and application of reliable, fair, and trustworthy artificial intelligence prediction models.