Wang Hao, Alanis Naomi, Haygood Laura, Swoboda Thomas K, Hoot Nathan, Phillips Daniel, Knowles Heidi, Stinson Sara Ann, Mehta Prachi, Sambamoorthi Usha
Department of Emergency Medicine, JPS Health Network, Fort Worth, Texas, USA.
Health Sciences Librarian for Public Health, Brown University, Providence, Rhode Island, USA.
Acad Emerg Med. 2024 Jul;31(7):696-706. doi: 10.1111/acem.14937. Epub 2024 May 16.
Natural language processing (NLP) represents one of the adjunct technologies within artificial intelligence and machine learning, creating structure out of unstructured data. This study aims to assess the performance of employing NLP to identify and categorize unstructured data within the emergency medicine (EM) setting.
We systematically searched publications related to EM research and NLP across databases including MEDLINE, Embase, Scopus, CENTRAL, and ProQuest Dissertations & Theses Global. Independent reviewers screened, reviewed, and evaluated article quality and bias. NLP usage was categorized into syndromic surveillance, radiologic interpretation, and identification of specific diseases/events/syndromes, with respective sensitivity analysis reported. Performance metrics for NLP usage were calculated and the overall area under the summary of receiver operating characteristic curve (SROC) was determined.
A total of 27 studies underwent meta-analysis. Findings indicated an overall mean sensitivity (recall) of 82%-87%, specificity of 95%, with the area under the SROC at 0.96 (95% CI 0.94-0.98). Optimal performance using NLP was observed in radiologic interpretation, demonstrating an overall mean sensitivity of 93% and specificity of 96%.
Our analysis revealed a generally favorable performance accuracy in using NLP within EM research, particularly in the realm of radiologic interpretation. Consequently, we advocate for the adoption of NLP-based research to augment EM health care management.
自然语言处理（NLP）是人工智能和机器学习中的辅助技术之一，可从非结构化数据中创建结构。本研究旨在评估在急诊医学（EM）环境中使用NLP识别和分类非结构化数据的性能。
我们系统地检索了包括MEDLINE、Embase、Scopus、CENTRAL和ProQuest Dissertations & Theses Global在内的数据库中与EM研究和NLP相关的出版物。独立评审员对文章质量和偏倚进行筛选、评审和评估。NLP的使用分为症状监测、放射学解释以及特定疾病/事件/综合征的识别，并报告了各自的敏感性分析。计算了NLP使用的性能指标，并确定了受试者操作特征曲线总结（SROC）下的总面积。
共有27项研究进行了荟萃分析。结果表明，总体平均敏感性（召回率）为82%-87%，特异性为95%，SROC下的面积为0.96（95%CI 0.94-0.98）。在放射学解释中观察到使用NLP的最佳性能，总体平均敏感性为93%，特异性为96%。
我们的分析表明，在EM研究中使用NLP通常具有良好的性能准确性，特别是在放射学解释领域。因此，我们主张采用基于NLP的研究来加强EM医疗保健管理。
