Institute of Health Informatics, University College London, London, United Kingdom.
Health Data Research UK, University College London, London, United Kingdom.
J Am Med Inform Assoc. 2021 Mar 18;28(4):791-800. doi: 10.1093/jamia/ocaa295.
Risk prediction models are widely used to inform evidence-based clinical decision making. However, few models developed from single cohorts can perform consistently well at population level where diverse prognoses exist (such as the SARS-CoV-2 [severe acute respiratory syndrome coronavirus 2] pandemic). This study aims at tackling this challenge by synergizing prediction models from the literature using ensemble learning.
In this study, we selected and reimplemented 7 prediction models for COVID-19 (coronavirus disease 2019) that were derived from diverse cohorts and used different implementation techniques. A novel ensemble learning framework was proposed to synergize them for realizing personalized predictions for individual patients. Four diverse international cohorts (2 from the United Kingdom and 2 from China; N = 5394) were used to validate all 8 models on discrimination, calibration, and clinical usefulness.
Results showed that individual prediction models could perform well on some cohorts while poorly on others. Conversely, the ensemble model achieved the best performances consistently on all metrics quantifying discrimination, calibration, and clinical usefulness. Performance disparities were observed in cohorts from the 2 countries: all models achieved better performances on the China cohorts.
When individual models were learned from complementary cohorts, the synergized model had the potential to achieve better performances than any individual model. Results indicate that blood parameters and physiological measurements might have better predictive powers when collected early, which remains to be confirmed by further studies.
Combining a diverse set of individual prediction models, the ensemble method can synergize a robust and well-performing model by choosing the most competent ones for individual patients.
风险预测模型被广泛用于为基于证据的临床决策提供信息。然而，很少有从单一队列开发的模型能够在存在多种预后的人群水平上表现一致（例如 SARS-CoV-2[严重急性呼吸综合征冠状病毒 2]大流行）。本研究旨在通过使用集成学习来协同利用文献中的预测模型来解决这一挑战。
在这项研究中，我们选择并重新实现了 7 种用于 COVID-19（2019 年冠状病毒病）的预测模型，这些模型来自不同的队列，并使用了不同的实现技术。提出了一种新的集成学习框架来协同它们，以便为个体患者实现个性化预测。使用来自 4 个不同国际队列（来自英国的 2 个队列和来自中国的 2 个队列；N=5394）验证了所有 8 种模型在区分度、校准和临床实用性方面的性能。
结果表明，个别预测模型在某些队列上表现良好，而在其他队列上表现不佳。相反，集成模型在所有衡量区分度、校准和临床实用性的指标上都始终表现出最佳性能。在来自这 2 个国家的队列中观察到了性能差异：所有模型在中国队列上都取得了更好的性能。
当从互补队列中学习个体模型时，协同模型有可能比任何个体模型都取得更好的性能。结果表明，当早期采集血液参数和生理测量值时，它们可能具有更好的预测能力，这有待进一步研究证实。
通过组合一组不同的个体预测模型，集成方法可以通过选择最适合个体患者的模型来协同构建一个稳健且性能良好的模型。
