Xie Qi, Wang Xinglei, Pei Juhong, Wu Yinping, Guo Qiang, Su Yujie, Yan Hui, Nan Ruiling, Chen Haixia, Dou Xinman
School of Nursing, Lanzhou University, Lanzhou, Gansu, China.
Department of Liver Diseases Branch, Lanzhou University Second Hospital, Lanzhou, Gansu, China.
J Am Med Dir Assoc. 2022 Oct;23(10):1655-1668.e6. doi: 10.1016/j.jamda.2022.06.020. Epub 2022 Jul 31.
To critically appraise and quantify the performance studies by employing machine learning (ML) to predict delirium.
A systematic review and meta-analysis.
Articles reporting the use of ML to predict delirium in adult patients were included. Studies were excluded if (1) the primary goal was only the identification of various risk factors for delirium; (2) the full-text article was not found; and (3) the article was published in a language other than English/Chinese.
PubMed, Embase, Cochrane Library database, Web of Science, Grey literature, and other relevant databases for the related publications were searched (from inception to December 15, 2021). The data were extracted using a standard checklist, and the risk of bias was assessed through the prediction model risk of bias assessment tool. Meta-analysis with the area under the receiver operating characteristic curve, sensitivity, and specificity as effect measures, was performed with Metadisc software. Cochran Q and I statistics were used to assess the heterogeneity. Meta-regression was performed to determine the potential effect of adjustment for the key covariates.
A total of 22 studies were included. Only 4 of 22 studies were quantitatively analyzed. The studies varied widely in reporting about the study participants, features and selection, handling of missing data, sample size calculations, and the intended clinical application of the model. For ML models, the overall pooled area under the receiver operating characteristic curve for predicting delirium was 0.89, sensitivity 0.85 (95% confidence interval 0.84‒0.85), and specificity 0.80 (95% confidence interval 0.81-0.80).
We found that the ML model showed excellent performance in predicting delirium. This review highlights the potential shortcomings of the current approaches, including low comparability and reproducibility. Finally, we present the various recommendations on how these challenges can be effectively addressed before deploying these models in prospective analyses.
通过运用机器学习（ML）预测谵妄来严格评估和量化相关性能研究。
系统评价和荟萃分析。
纳入报告使用ML预测成年患者谵妄的文章。若存在以下情况则排除研究：（1）主要目标仅是识别谵妄的各种风险因素；（2）未找到全文；（3）文章以非英文/中文发表。
检索了PubMed、Embase、Cochrane图书馆数据库、Web of Science、灰色文献及其他相关数据库以查找相关出版物（从创刊至2021年12月15日）。使用标准清单提取数据，并通过预测模型偏倚风险评估工具评估偏倚风险。采用Metadisc软件进行以受试者工作特征曲线下面积、敏感性和特异性为效应指标的荟萃分析。使用Cochran Q和I统计量评估异质性。进行荟萃回归以确定对关键协变量进行调整的潜在效应。
共纳入22项研究。22项研究中仅有4项进行了定量分析。这些研究在研究参与者、特征与选择、缺失数据处理、样本量计算以及模型的预期临床应用等方面的报告差异很大。对于ML模型，预测谵妄的受试者工作特征曲线下总体合并面积为0.89，敏感性为0.85（95%置信区间0.84‒0.85），特异性为0.80（95%置信区间0.81 - 0.80）。
我们发现ML模型在预测谵妄方面表现出色。本综述突出了当前方法的潜在缺点，包括可比性和可重复性低。最后，我们针对在将这些模型用于前瞻性分析之前如何有效应对这些挑战提出了各种建议。
