Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
J Am Med Inform Assoc. 2023 Jan 18;30(2):348-354. doi: 10.1093/jamia/ocac221.
Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.
在严重急性呼吸综合征冠状病毒 2 (SARS-CoV-2) 大流行期间，医疗保健利用的突然变化可能影响了在大流行之前训练的临床预测模型的性能。在这项研究中，我们评估了一种机器学习、基于电子健康记录的死亡率预测算法的性能随时间的变化，该算法目前在临床实践中用于识别可能受益于早期预先护理计划对话的癌症患者。我们表明，在大流行期间，算法识别高危患者的能力大幅且持续下降。大流行高峰期实验室利用的减少可能导致了偏差。在大流行期间，校准和整体判别能力并没有明显下降。这就要求对使用大流行期间输入的预测算法的性能和再培训给予谨慎关注。
