Baron Jason M, Huang Richard, McEvoy Dustin, Dighe Anand S
Department of Pathology, Massachusetts General Hospital, Boston, Massachusetts, USA.
Havard Medical School, Boston, Massachusetts, USA.
JAMIA Open. 2021 Mar 1;4(1):ooab006. doi: 10.1093/jamiaopen/ooab006. eCollection 2021 Jan.
While well-designed clinical decision support (CDS) alerts can improve patient care, utilization management, and population health, excessive alerting may be counterproductive, leading to clinician burden and alert fatigue. We sought to develop machine learning models to predict whether a clinician will accept the advice provided by a CDS alert. Such models could reduce alert burden by targeting CDS alerts to specific cases where they are most likely to be effective.
We focused on a set of laboratory test ordering alerts, deployed at 8 hospitals within the Partners Healthcare System. The alerts notified clinicians of duplicate laboratory test orders and advised discontinuation. We captured key attributes surrounding 60 399 alert firings, including clinician and patient variables, and whether the clinician complied with the alert. Using these data, we developed logistic regression models to predict alert compliance.
We identified key factors that predicted alert compliance; for example, clinicians were less likely to comply with duplicate test alerts triggered in patients with a prior abnormal result for the test or in the context of a nonvisit-based encounter (eg, phone call). Likewise, differences in practice patterns between clinicians appeared to impact alert compliance. Our best-performing predictive model achieved an area under the receiver operating characteristic curve (AUC) of 0.82. Incorporating this model into the alerting logic could have averted more than 1900 alerts at a cost of fewer than 200 additional duplicate tests.
Deploying predictive models to target CDS alerts may substantially reduce clinician alert burden while maintaining most or all the CDS benefit.
虽然设计良好的临床决策支持（CDS）警报可以改善患者护理、利用管理和人群健康，但过度警报可能会适得其反，导致临床医生负担加重和警报疲劳。我们试图开发机器学习模型，以预测临床医生是否会接受CDS警报提供的建议。这样的模型可以通过将CDS警报针对最有可能有效的特定病例，来减轻警报负担。
我们聚焦于一组实验室检查医嘱警报，这些警报在合作伙伴医疗系统内的8家医院部署。这些警报会通知临床医生重复的实验室检查医嘱，并建议停止。我们收集了围绕60399次警报触发的关键属性，包括临床医生和患者变量，以及临床医生是否遵守警报。利用这些数据，我们开发了逻辑回归模型来预测警报遵守情况。
我们确定了预测警报遵守情况的关键因素；例如，临床医生不太可能遵守在之前该检查结果异常的患者中或在非就诊情况下（如电话）触发的重复检查警报。同样，临床医生之间的实践模式差异似乎也会影响警报遵守情况。我们表现最佳的预测模型在受试者工作特征曲线（AUC）下的面积为0.82。将此模型纳入警报逻辑中，可能以少于200次额外重复检查的成本避免超过1900次警报。
部署预测模型来针对CDS警报，可能会在保持大部分或所有CDS益处的同时，大幅减轻临床医生的警报负担。
