Microsoft Research, Cambridge, UK.
Department of Anaesthesiology and Pain Medicine, University of Washington, Seattle, WA, USA.
Br J Anaesth. 2022 Apr;128(4):623-635. doi: 10.1016/j.bja.2021.10.052. Epub 2021 Dec 17.
Postoperative hypotension is associated with adverse outcomes, but intraoperative prediction of postanaesthesia care unit (PACU) hypotension is not routine in anaesthesiology workflow. Although machine learning models may support clinician prediction of PACU hypotension, clinician acceptance of prediction models is poorly understood.
We developed a clinically informed gradient boosting machine learning model using preoperative and intraoperative data from 88 446 surgical patients from 2015 to 2019. Nine anaesthesiologists each made 192 predictions of PACU hypotension using a web-based visualisation tool with and without input from the machine learning model. Questionnaires and interviews were analysed using thematic content analysis for model acceptance by anaesthesiologists.
The model predicted PACU hypotension in 17 029 patients (area under the receiver operating characteristic [AUROC] 0.82 [95% confidence interval {CI}: 0.81-0.83] and average precision 0.40 [95% CI: 0.38-0.42]). On a random representative subset of 192 cases, anaesthesiologist performance improved from AUROC 0.67 (95% CI: 0.60-0.73) to AUROC 0.74 (95% CI: 0.68-0.79) with model predictions and information on risk factors. Anaesthesiologists perceived more value and expressed trust in the prediction model for prospective planning, informing PACU handoffs, and drawing attention to unexpected cases of PACU hypotension, but they doubted the model when predictions and associated features were not aligned with clinical judgement. Anaesthesiologists expressed interest in patient-specific thresholds for defining and treating postoperative hypotension.
The ability of anaesthesiologists to predict PACU hypotension was improved by exposure to machine learning model predictions. Clinicians acknowledged value and trust in machine learning technology. Increasing familiarity with clinical use of model predictions is needed for effective integration into perioperative workflows.
术后低血压与不良结局相关,但麻醉学工作流程中并未常规预测麻醉后恢复室(PACU)低血压。尽管机器学习模型可能支持临床医生预测 PACU 低血压,但对临床医生对预测模型的接受程度了解甚少。
我们使用 2015 年至 2019 年间 88446 例手术患者的术前和术中数据,开发了一种具有临床意义的梯度提升机机器学习模型。9 名麻醉医生每人使用基于网络的可视化工具进行了 192 次 PACU 低血压预测,其中包括和不包括机器学习模型的输入。使用主题内容分析对调查问卷和访谈进行分析,以了解麻醉医生对模型的接受程度。
该模型预测了 17029 例 PACU 低血压患者(受试者工作特征曲线下面积[AUROC]为 0.82[95%置信区间{CI}:0.81-0.83],平均精度为 0.40[95%CI:0.38-0.42])。在随机代表性的 192 例病例中,麻醉医生的表现从 AUROC 0.67(95%CI:0.60-0.73)提高到 AUROC 0.74(95%CI:0.68-0.79),同时还提供了模型预测和风险因素信息。麻醉医生认为模型在预测前瞻性规划、通知 PACU 交接和引起对 PACU 低血压意外病例的注意方面更有价值,也更值得信任,但当预测结果与临床判断不一致时,他们会对模型产生怀疑。麻醉医生对定义和治疗术后低血压的患者特定阈值表示了兴趣。
暴露于机器学习模型预测可提高麻醉医生预测 PACU 低血压的能力。临床医生承认机器学习技术的价值和信任。需要提高对模型预测的临床应用的熟悉程度,以便将其有效地整合到围手术期工作流程中。