Wang Zichen, Wang Wen, Sun Che, Li Jili, Xie Shuangyi, Xu Jiayue, Zou Kang, Jin Yinghui, Yan Siyu, Liao Xuelian, Kang Yan, Coopersmith Craig M, Sun Xin
Department of Critical Care Medicine, Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, 610041, China.
NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu, 610041, China.
NPJ Digit Med. 2025 Apr 7;8(1):190. doi: 10.1038/s41746-025-01587-1.
Sepsis real-time prediction models (SRPMs) provide timely alerts and may improve patient outcomes but face limited clinical adoption due to inconsistent validation methods and potential biases. Comprehensive evaluation, including external full-window validation with model- and outcome-level metrics, is crucial for real-world effectiveness, yet performance evidence remains scarce. This study systematically reviewed SRPM performance across validation methods, analyzing 91 studies from multiple databases. Only 54.9% applied full-window validation with both metric types. Performance decreased under external and full-window validation, with median AUROCs of 0.886 and 0.861 at 6- and 12-hours pre-onset, dropping to 0.783 in full-window external validation. Median Utility Scores declined from 0.381 in internal to -0.164 in external validation. Combining AUROC and Utility Score identified top-performing SRPMs in 18.7% of studies. Hand-crafted features significantly improved performance. Future research should focus on multi-center datasets, hand-crafted features, multi-metric full-window validation, and prospective trials to support clinical implementation.
脓毒症实时预测模型（SRPMs）可提供及时警报，可能改善患者预后，但由于验证方法不一致和潜在偏差，其临床应用受限。全面评估，包括使用模型和结局层面指标进行外部全窗口验证，对于实际有效性至关重要，但性能证据仍然稀缺。本研究系统回顾了不同验证方法下SRPMs的性能，分析了来自多个数据库的91项研究。只有54.9%的研究同时使用两种指标类型进行全窗口验证。在外部和全窗口验证下，性能有所下降，发病前6小时和12小时的中位受试者工作特征曲线下面积（AUROCs）分别为0.886和0.861，在全窗口外部验证中降至0.783。中位效用得分从内部验证的0.381降至外部验证的-0.164。在18.7%的研究中，结合AUROC和效用得分确定了表现最佳的SRPMs。手工制作的特征显著提高了性能。未来的研究应侧重于多中心数据集、手工制作的特征、多指标全窗口验证和前瞻性试验，以支持临床应用。
