Mathiszig-Lee Jakob F, Catling Finneas J R, Moonesinghe S Ramani, Brett Stephen J
Department of Surgery and Cancer, Imperial College London, London, UK.
Department of Anaesthesia and Perioperative Medicine, Royal Marsden Hospital, London, UK.
NPJ Digit Med. 2022 Jun 8;5(1):70. doi: 10.1038/s41746-022-00616-7.
Clinical prediction models typically make point estimates of risk. However, values of key variables are often missing during model development or at prediction time, meaning that the point estimates mask significant uncertainty and can lead to over-confident decision making. We present a model of mortality risk in emergency laparotomy which instead presents a distribution of predicted risks, highlighting the uncertainty over the risk of death with an intuitive visualisation. We developed and validated our model using data from 127134 emergency laparotomies from patients in England and Wales during 2013-2019. We captured the uncertainty arising from missing data using multiple imputation, allowing prospective, patient-specific imputation for variables that were frequently missing. Prospective imputation allows early prognostication in patients where these variables are not yet measured, accounting for the additional uncertainty this induces. Our model showed good discrimination and calibration (95% confidence intervals: Brier score 0.071-0.078, C statistic 0.859-0.873, calibration error 0.031-0.059) on unseen data from 37 hospitals, consistently improving upon the current gold-standard model. The dispersion of the predicted risks varied significantly between patients and increased where prospective imputation occurred. We present a case study that illustrates the potential impact of uncertainty quantification on clinical decision making. Our model improves mortality risk prediction in emergency laparotomy and has the potential to inform decision-makers and assist discussions with patients and their families. Our analysis code was robustly developed and is publicly available for easy replication of our study and adaptation to predicting other outcomes.
临床预测模型通常会给出风险的点估计值。然而，在模型开发过程中或预测时，关键变量的值常常缺失，这意味着点估计掩盖了显著的不确定性，并可能导致过度自信的决策。我们提出了一种急诊剖腹手术死亡率风险模型，该模型呈现的是预测风险的分布，通过直观的可视化突出了死亡风险的不确定性。我们使用2013年至2019年期间英格兰和威尔士患者的127134例急诊剖腹手术数据开发并验证了我们的模型。我们使用多重填补法捕捉因数据缺失而产生的不确定性，允许对经常缺失的变量进行前瞻性的、针对患者的填补。前瞻性填补允许在尚未测量这些变量的患者中进行早期预后评估，同时考虑到由此引发的额外不确定性。我们的模型在来自37家医院的未见数据上显示出良好的区分度和校准度（95%置信区间：Brier评分0.071 - 0.078，C统计量0.859 - 0.873，校准误差0.031 - 0.059），持续优于当前的金标准模型。预测风险的离散度在患者之间差异显著，在前瞻性填补的情况下会增加。我们展示了一个案例研究，说明了不确定性量化对临床决策的潜在影响。我们的模型改进了急诊剖腹手术中的死亡率风险预测，有可能为决策者提供信息，并有助于与患者及其家属进行讨论。我们的分析代码经过稳健开发，可公开获取，便于复制我们的研究并适用于预测其他结果。
