Department of Surgery, The Queen Elizabeth Hospital, The University of Adelaide, Adelaide, South Australia, Australia; Royal Australasian College of Surgeons, Adelaide, South Australia, Australia; Health and Information, Adelaide, South Australia, Australia. Electronic address: https://twitter.com/josh.kovoor.
Health and Information, Adelaide, South Australia, Australia; Royal Adelaide Hospital, Adelaide, South Australia, Australia; University of Adelaide, Adelaide, South Australia, Australia.
Surgery. 2023 Dec;174(6):1309-1314. doi: 10.1016/j.surg.2023.08.021. Epub 2023 Sep 29.
This study aimed to examine the accuracy with which multiple natural language processing artificial intelligence models could predict discharge and readmissions after general surgery.
Natural language processing models were derived and validated to predict discharge within the next 48 hours and 7 days and readmission within 30 days (based on daily ward round notes and discharge summaries, respectively) for general surgery inpatients at 2 South Australian hospitals. Natural language processing models included logistic regression, artificial neural networks, and Bidirectional Encoder Representations from Transformers.
For discharge prediction analyses, 14,690 admissions were included. For readmission prediction analyses, 12,457 patients were included. For prediction of discharge within 48 hours, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.86 and 0.86 for Bidirectional Encoder Representations from Transformers, 0.82 and 0.81 for logistic regression, and 0.82 and 0.81 for artificial neural networks. For prediction of discharge within 7 days, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.82 and 0.81 for Bidirectional Encoder Representations from Transformers, 0.75 and 0.72 for logistic regression, and 0.68 and 0.67 for artificial neural networks. For readmission prediction within 30 days, derivation and validation data set area under the receiver operator characteristic curves were, respectively: 0.55 and 0.59 for Bidirectional Encoder Representations from Transformers and 0.77 and 0.62 for logistic regression.
Modern natural language processing models, particularly Bidirectional Encoder Representations from Transformers, can effectively and accurately identify general surgery patients who will be discharged in the next 48 hours. However, these approaches are less capable of identifying general surgery patients who will be discharged within the next 7 days or who will experience readmission within 30 days of discharge.
本研究旨在检验多种自然语言处理人工智能模型预测普外科患者出院和再入院的准确性。
在南澳大利亚的 2 家医院，基于每日病房查房记录和出院小结，分别针对普外科住院患者，采用逻辑回归、人工神经网络和基于 Transformer 的双向编码器表示等自然语言处理模型，提取并验证了预测患者在接下来 48 小时内和 7 天内出院以及在 30 天内再入院的模型。
在出院预测分析中，共纳入 14690 例入院患者。在再入院预测分析中，共纳入 12457 例患者。对于 48 小时内的出院预测，在推导和验证数据集的受试者工作特征曲线下面积分别为：基于 Transformer 的双向编码器表示为 0.86 和 0.86，逻辑回归为 0.82 和 0.81，人工神经网络为 0.82 和 0.81。对于 7 天内的出院预测，在推导和验证数据集的受试者工作特征曲线下面积分别为：基于 Transformer 的双向编码器表示为 0.82 和 0.81，逻辑回归为 0.75 和 0.72，人工神经网络为 0.68 和 0.67。对于 30 天内的再入院预测，在推导和验证数据集的受试者工作特征曲线下面积分别为：基于 Transformer 的双向编码器表示为 0.55 和 0.59，逻辑回归为 0.77 和 0.62。
现代自然语言处理模型，特别是基于 Transformer 的双向编码器表示，可以有效地、准确地识别出将在接下来 48 小时内出院的普外科患者。然而，这些方法在识别将在接下来 7 天内出院或在出院后 30 天内再入院的普外科患者方面能力较弱。
