Division of Plastic Surgery, Department of Surgery, University of Pennsylvania. Philadelphia, PA.
Department of Plastic and Reconstructive Surgery, Brown University. Providence, RI.
Ann Surg. 2021 May 1;273(5):900-908. doi: 10.1097/SLA.0000000000004419.
The aim of this study was to systematically assess the application and potential benefits of natural language processing (NLP) in surgical outcomes research.
Widespread implementation of electronic health records (EHRs) has generated a massive patient data source. Traditional methods of data capture, such as billing codes and/or manual review of free-text narratives in EHRs, are highly labor-intensive, costly, subjective, and potentially prone to bias.
A literature search of PubMed, MEDLINE, Web of Science, and Embase identified all articles published starting in 2000 that used NLP models to assess perioperative surgical outcomes. Evaluation metrics of NLP systems were assessed by means of pooled analysis and meta-analysis. Qualitative synthesis was carried out to assess the results and risk of bias on outcomes.
The present study included 29 articles, with over half (n = 15) published after 2018. The most common outcome identified using NLP was postoperative complications (n = 14). Compared to traditional non-NLP models, NLP models identified postoperative complications with higher sensitivity [0.92 (0.87-0.95) vs 0.58 (0.33-0.79), P < 0.001]. The specificities were comparable at 0.99 (0.96-1.00) and 0.98 (0.95-0.99), respectively. Using summary of likelihood ratio matrices, traditional non-NLP models have clinical utility for confirming documentation of outcomes/diagnoses, whereas NLP models may be reliably utilized for both confirming and ruling out documentation of outcomes/diagnoses.
NLP usage to extract a range of surgical outcomes, particularly postoperative complications, is accelerating across disciplines and areas of clinical outcomes research. NLP and traditional non-NLP approaches demonstrate similar performance measures, but NLP is superior in ruling out documentation of surgical outcomes.
本研究旨在系统评估自然语言处理（NLP）在手术结果研究中的应用和潜在益处。
电子健康记录（EHR）的广泛应用产生了大量的患者数据来源。传统的数据采集方法，如计费代码和/或 EHR 中自由文本叙述的手动审查，非常耗费人力、昂贵、主观，并且可能存在偏差。
通过对 PubMed、MEDLINE、Web of Science 和 Embase 进行文献检索，确定了所有自 2000 年以来使用 NLP 模型评估围手术期手术结果的文章。通过汇总分析和荟萃分析评估 NLP 系统的评估指标。进行定性综合评估以评估结果和偏倚风险。
本研究共纳入 29 篇文章，其中超过一半（n = 15）发表于 2018 年之后。使用 NLP 识别的最常见结果是术后并发症（n = 14）。与传统的非 NLP 模型相比，NLP 模型识别术后并发症的敏感性更高[0.92（0.87-0.95）比 0.58（0.33-0.79），P < 0.001]。特异性也相当，分别为 0.99（0.96-1.00）和 0.98（0.95-0.99）。使用似然比矩阵的总结，传统的非 NLP 模型对于确认结局/诊断的记录具有临床实用性，而 NLP 模型可用于确认和排除结局/诊断的记录。
NLP 在提取一系列手术结果中的使用正在加速，特别是在术后并发症方面，在各学科和临床结果研究领域都有应用。NLP 和传统的非 NLP 方法表现出相似的性能指标，但 NLP 在排除手术结果的记录方面更优。
