A doctor using a risk calculator to decide how aggressively to treat a patient with chronic kidney disease may be relying on predictions that are years out of date. Many prognostic models—the algorithms that estimate a patient’s risk of a future health event—are built on data from patients treated a decade or more ago. Clinical practice changes, and outcomes improve, so those old models systematically overestimate risk for today’s patients. The researchers have developed a method called temporal recalibration that adjusts predictions to reflect current conditions. This project will extend that method to handle competing risks (for example, a patient dying from something else before the predicted event occurs), validate it properly, and produce free software and tutorials so other researchers can use it. If successful, the work will make risk predictions more accurate across many clinical settings—not just kidney disease but any condition where prognostic models guide treatment decisions. Better risk estimates mean fewer unnecessary treatments for patients who are actually at low risk, and more appropriate monitoring for those who need it. The software and teaching materials will be freely available, lowering the barrier for uptake by clinicians and researchers worldwide.
View original technical description
Research question Can we maximise patient benefit from prognostic models using temporal recalibration methods to improve the accuracy of predictions for new patients? Background For an individual with a given health condition (e.g. chronic kidney disease CKD), a prognostic model estimates their risk of a future health outcome and can be used to inform patient monitoring and treatment decisions. A major limitation of many prognostic models is that they are developed from datasets from an earlier time-period (e.g. 2000-2010) which may not represent current (e.g. 2024) clinical care and outcome risks. Any changes in clinical practice or improvements in prognosis will lead to out-dated predictions for individuals diagnosed today. Another limitation is not appropriately accounting for competing risks when developing and validating prognostic models. To address these issues, we have developed novel methodology for prognostic model development, called temporal recalibration, that ensures that predictions are as up-to-date as possible. While the temporal recalibration approach has been shown to improve the accuracy of model predictions, further methodological extensions are required. Aims and Objectives Our overarching objective is to maximise patient benefit from prognostic models by extending and making temporal recalibration methods easily accessible to improve the accuracy of predictions for new patients (with an applied example in CKD). Specifically, we aim to: Determine how best to internally validate models developed using temporal recalibration, with and without competing risks. Develop software and tutorials to aid with the uptake of temporal recalibration methods. Methods We will provide guidance on how to combine temporal recalibration with bootstrapping and cross-validation methods to provide appropriate methodology for internal validation. These approaches will be combined with subsequent optimism-adjustment and evaluated through simulation studies. The process will be repeated across a broad range of scenarios, varying the magnitude of calibration drift, number of included predictors, total participants, and number of events, as well as allowing for competing risks and missing data. Finally, we will develop software and tutorials to aid other researchers to apply the developed methodology going forwards. Timelines for delivery The 12-month project will begin in March 2025, beginning with work focusing on the internal validation of temporally recalibrated models. The methods development work is expected to be completed by September and the applied example of the methods in CKD will take place in the final 3 months of the project. Work on the software development aspect is planned for August-November which will lead on to the development of tutorials and webtools in October-January. Anticipated Impact and Dissemination We will publish our methodological extensions and exemplars in academic journals and disseminate our findings at international conferences and our dissemination workshop. The software and teaching examples will be made freely available on www.prognosisresearch.com. The overall impact of the project will be to ensure more appropriate estimates of risk are given across many clinical applications, leading to better-informed treatment decision-making and patient management across a wide range of settings.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know