Department of Anesthesiology & Pain Medicine, University of Washington, 1959 NE Pacific Street, BB-1469, Box 356540, Seattle, WA, 98195-6540, USA.
Department of Biomedical & Health Informatics, University of Washington, 850 Republican Street, Box 358047, Seattle, WA, 98109, USA.
BMC Anesthesiol. 2023 Sep 4;23(1):296. doi: 10.1186/s12871-023-02248-0.
Electronic health records (EHR) contain large volumes of unstructured free-form text notes that richly describe a patient's health and medical comorbidities. It is unclear if perioperative risk stratification can be performed directly from these notes without manual data extraction. We conduct a feasibility study using natural language processing (NLP) to predict the American Society of Anesthesiologists Physical Status Classification (ASA-PS) as a surrogate measure for perioperative risk. We explore prediction performance using four different model types and compare the use of different note sections versus the whole note. We use Shapley values to explain model predictions and analyze disagreement between model and human anesthesiologist predictions.
Single-center retrospective cohort analysis of EHR notes from patients undergoing procedures with anesthesia care spanning all procedural specialties during a 5 year period who were not assigned ASA VI and also had a preoperative evaluation note filed within 90 days prior to the procedure. NLP models were trained for each combination of 4 models and 8 text snippets from notes. Model performance was compared using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Shapley values were used to explain model predictions. Error analysis and model explanation using Shapley values was conducted for the best performing model.
Final dataset includes 38,566 patients undergoing 61,503 procedures with anesthesia care. Prevalence of ASA-PS was 8.81% for ASA I, 31.4% for ASA II, 43.25% for ASA III, and 16.54% for ASA IV-V. The best performing models were the BioClinicalBERT model on the truncated note task (macro-average AUROC 0.845) and the fastText model on the full note task (macro-average AUROC 0.865). Shapley values reveal human-interpretable model predictions. Error analysis reveals that some original ASA-PS assignments may be incorrect and the model is making a reasonable prediction in these cases.
Text classification models can accurately predict a patient's illness severity using only free-form text descriptions of patients without any manual data extraction. They can be an additional patient safety tool in the perioperative setting and reduce manual chart review for medical billing. Shapley feature attributions produce explanations that logically support model predictions and are understandable to clinicians.
电子健康记录 (EHR) 包含大量非结构化的自由格式文本注释，这些注释丰富地描述了患者的健康状况和合并症。目前尚不清楚是否可以直接从这些注释中进行围手术期风险分层，而无需手动提取数据。我们使用自然语言处理 (NLP) 进行了一项可行性研究，以预测美国麻醉师协会身体状况分类 (ASA-PS) 作为围手术期风险的替代指标。我们探索了使用四种不同模型类型的预测性能，并比较了使用不同注释部分与使用整个注释的情况。我们使用 Shapley 值来解释模型预测，并分析模型预测与人类麻醉师预测之间的差异。
这是一项单中心回顾性队列分析，涉及在 5 年内接受麻醉护理的患者的 EHR 注释，这些患者接受了各种手术，且没有被分配 ASA VI，并且在手术前 90 天内也有术前评估注释。为每个组合的 4 个模型和 8 个注释片段训练了 NLP 模型。使用接收者操作特征曲线下面积 (AUROC) 和精度召回曲线下面积 (AUPRC) 比较模型性能。使用 Shapley 值来解释模型预测。对表现最佳的模型进行错误分析和 Shapley 值模型解释。
最终数据集包括 38566 名患者，接受了 61503 次麻醉护理手术。ASA-PS 的患病率为 ASA I 为 8.81%，ASA II 为 31.4%，ASA III 为 43.25%，ASA IV-V 为 16.54%。表现最佳的模型是在截断注释任务上的 BioClinicalBERT 模型（宏观平均 AUROC 为 0.845）和在完整注释任务上的 fastText 模型（宏观平均 AUROC 为 0.865）。Shapley 值揭示了可解释的模型预测。错误分析表明，一些原始的 ASA-PS 分配可能不正确，并且在这些情况下模型做出了合理的预测。
文本分类模型可以仅使用患者的自由格式文本描述来准确预测患者的疾病严重程度，而无需任何手动数据提取。它们可以成为围手术期的附加患者安全工具，并减少医疗计费的手动图表审查。Shapley 特征归因生成了逻辑上支持模型预测且临床医生易于理解的解释。
