External validation of prognostic models: what, why, how, when and where?

2020/09/03 by Chava L. Ramspek, Kitty J. Jager, Friedo W. Dekker +2 · 12 citations
Decision Sciences · Economics, Econometrics and Finance · Mathematics · #Health Systems, Economic Evaluations, Quality of Life #Meta-analysis and systematic reviews #Statistical Methods in Clinical Trials

paper · pdf · doi:10.1093/ckj/sfaa188

openalex publication_date 2020/09/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model's reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.

Citations

• External validation is necessary in prediction research:
• Prediction models need appropriate internal, internal–external, and external validation
• Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
• A new framework to enhance the interpretation of external validation studies of clinical prediction models
• Sample size considerations for the external validation of a multivariable prognostic model: a resampling study
• Minimum sample size for developing a multivariable prediction model: PART II ‐ binary and time‐to‐event outcomes
• PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies

Cited by

• Using the cancer aging and research group- breast cancer (CARG-BC) predictive model in older adults (OA) with early breast cancer: an external validation study
• Automated Deep Learning–based Segmentation of the Dentate Nucleus Using Quantitative Susceptibility Mapping MRI
• Machine learning for endoscopic third ventriculostomy success prediction—a systematic review and meta-analysis
• Novel machine learning applications in peripartum care: a scoping review
• International validation of a pre-transplant risk assessment tool for graft survival in pediatric kidney transplant recipients
• When the whole is greater than the sum of its parts: why machine learning and conventional statistics are complementary for predicting future health outcomes
• AI in Rehabilitation Medicine: Opportunities and Challenges
• Predicting Chest Radiograph Findings from Electrocardiograms Using Interpretable Machine Learning
• Prediction of post-stroke urinary tract infection risk in immobile patients using machine learning: an observational cohort study
• External validation of GDM risk prediction models using a machine learning reciprocal model-exchange framework
• Technology-based forecasting approach for recognizing trade-off between time-to-market reduction and devising a scheduling process in open innovation management
• When impact trials are not feasible: alternatives to study the impact of prediction models on clinical practice

Related

• Building Reality Checks into the Translational Pathway for Diagnostic and Prognostic Models
• A new framework to enhance the interpretation of external validation studies of clinical prediction models
• There is no such thing as a validated prediction model
• Sample size considerations for the external validation of a multivariable prognostic model: a resampling study
• Empirical Evidence That There Is No Such Thing As A Validated Prediction Model