2024/12/13 by Arthur Chatton, Émilie Pilote, Chatton, Arthur +9
Computer Science · #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2412.10252
Objective: This study sought to compare the drop in predictive performance\nover time according to the modeling approach (regression versus machine\nlearning) used to build a kidney transplant failure prediction model with a\ntime-to-event outcome.\n Study Design and Setting: The Kidney Transplant Failure Score (KTFS) was used\nas a benchmark. We reused the data from which it was developed (DIVAT cohort,\nn=2,169) to build another prediction algorithm using a survival super learner\ncombining (semi-)parametric and non-parametric methods. Performance in DIVAT\nwas estimated for the two prediction models using internal validation. Then,\nthe drop in predictive performance was evaluated in the same geographical\npopulation approximately ten years later (EKiTE cohort, n=2,329).\n Results: In DIVAT, the super learner achieved better discrimination than the\nKTFS, with a tAUROC of 0.83 (0.79-0.87) compared to 0.76 (0.70-0.82). While the\ndiscrimination remained stable for the KTFS, it was not the case for the super\nlearner, with a drop to 0.80 (0.76-0.83). Regarding calibration, the survival\nSL overestimated graft survival at development, while the KTFS underestimated\ngraft survival ten years later. Brier score values were similar regardless of\nthe approach and the timing.\n Conclusion: The more flexible SL provided superior discrimination on the\npopulation used to fit it compared to a Cox model and similar discrimination\nwhen applied to a future dataset of the same population. Both methods are\nsubject to calibration drift over time. However, weak calibration on the\npopulation used to develop the prediction model was correct only for the Cox\nmodel, and recalibration should be considered in the future to correct the\ncalibration drift.\n