Currently a medical student at Emory University School of Medicine, Atlanta, Georgia.
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta.
JAMA Netw Open. 2023 Jul 3;6(7):e2322299. doi: 10.1001/jamanetworkopen.2023.22299.
Natural language processing (NLP) has the potential to enable faster treatment access by reducing clinician response time and improving electronic health record (EHR) efficiency.
To develop an NLP model that can accurately classify patient-initiated EHR messages and triage COVID-19 cases to reduce clinician response time and improve access to antiviral treatment.
DESIGN, SETTING, AND PARTICIPANTS: This retrospective cohort study assessed development of a novel NLP framework to classify patient-initiated EHR messages and subsequently evaluate the model's accuracy. Included patients sent messages via the EHR patient portal from 5 Atlanta, Georgia, hospitals between March 30 and September 1, 2022. Assessment of the model's accuracy consisted of manual review of message contents to confirm the classification label by a team of physicians, nurses, and medical students, followed by retrospective propensity score-matched clinical outcomes analysis.
Prescription of antiviral treatment for COVID-19.
The 2 primary outcomes were (1) physician-validated evaluation of the NLP model's message classification accuracy and (2) analysis of the model's potential clinical effect via increased patient access to treatment. The model classified messages into COVID-19-other (pertaining to COVID-19 but not reporting a positive test), COVID-19-positive (reporting a positive at-home COVID-19 test result), and non-COVID-19 (not pertaining to COVID-19).
Among 10 172 patients whose messages were included in analyses, the mean (SD) age was 58 (17) years; 6509 patients (64.0%) were women and 3663 (36.0%) were men. In terms of race and ethnicity, 2544 patients (25.0%) were African American or Black, 20 (0.2%) were American Indian or Alaska Native, 1508 (14.8%) were Asian, 28 (0.3%) were Native Hawaiian or other Pacific Islander, 5980 (58.8%) were White, 91 (0.9%) were more than 1 race or ethnicity, and 1 (0.01%) chose not to answer. The NLP model had high accuracy and sensitivity, with a macro F1 score of 94% and sensitivity of 85% for COVID-19-other, 96% for COVID-19-positive, and 100% for non-COVID-19 messages. Among the 3048 patient-generated messages reporting positive SARS-CoV-2 test results, 2982 (97.8%) were not documented in structured EHR data. Mean (SD) message response time for COVID-19-positive patients who received treatment (364.10 [784.47] minutes) was faster than for those who did not (490.38 [1132.14] minutes; P = .03). Likelihood of antiviral prescription was inversely correlated with message response time (odds ratio, 0.99 [95% CI, 0.98-1.00]; P = .003).
In this cohort study of 2982 COVID-19-positive patients, a novel NLP model classified patient-initiated EHR messages reporting positive COVID-19 test results with high sensitivity. Furthermore, when responses to patient messages occurred faster, patients were more likely to receive antiviral medical prescription within the 5-day treatment window. Although additional analysis on the effect on clinical outcomes is needed, these findings represent a possible use case for integration of NLP algorithms into clinical care.
自然语言处理（NLP）有可能通过减少临床医生的响应时间和提高电子健康记录（EHR）的效率来加速治疗的获得。
开发一种能够准确分类患者发起的 EHR 消息的 NLP 模型，以筛选 COVID-19 病例，从而减少临床医生的响应时间并改善获得抗病毒治疗的机会。
设计、地点和参与者：这是一项回顾性队列研究，评估了一种新的 NLP 框架的开发，以分类患者发起的 EHR 消息，并随后评估该模型的准确性。纳入了 2022 年 3 月 30 日至 9 月 1 日期间，来自佐治亚州亚特兰大的 5 家医院的患者通过 EHR 患者门户发送的消息。模型准确性的评估包括通过一组医生、护士和医学生对消息内容进行手动审查，以确认分类标签，随后进行回顾性倾向评分匹配的临床结果分析。
COVID-19 的抗病毒治疗处方。
主要结果有两个：（1）医生验证的 NLP 模型消息分类准确性评估；（2）通过增加患者获得治疗的机会，分析模型的潜在临床效果。该模型将消息分类为 COVID-19-其他（与 COVID-19 相关但未报告阳性检测结果）、COVID-19-阳性（报告家庭 COVID-19 检测结果阳性）和非 COVID-19（与 COVID-19 无关）。
在 10772 名其消息被纳入分析的患者中，平均（标准差）年龄为 58（17）岁；6509 名患者（64.0%）为女性，3663 名（36.0%）为男性。在种族和民族方面，2544 名患者（25.0%）为非裔美国人或黑人，20 名（0.2%）为美洲印第安人或阿拉斯加原住民，1508 名（14.8%）为亚洲人，28 名（0.3%）为夏威夷原住民或其他太平洋岛民，5980 名（58.8%）为白人，91 名（0.9%）为多种族或民族，1 名（0.01%）选择不回答。该 NLP 模型具有较高的准确性和敏感性，COVID-19-其他的宏观 F1 得分为 94%，敏感性为 85%，COVID-19-阳性的得分为 96%，非 COVID-19 的得分为 100%。在 3048 名报告 SARS-CoV-2 检测结果阳性的患者生成的消息中，有 2982 条（97.8%）未在结构化 EHR 数据中记录。接受治疗的 COVID-19-阳性患者的平均（标准差）消息响应时间为 364.10（784.47）分钟，比未接受治疗的患者（490.38（1132.14）分钟）更快（P=0.03）。抗病毒药物处方的可能性与消息响应时间呈负相关（比值比，0.99[95%置信区间，0.98-1.00]；P=0.003）。
在这项针对 2982 名 COVID-19 阳性患者的队列研究中，一种新的 NLP 模型以高灵敏度对报告 COVID-19 阳性检测结果的患者发起的 EHR 消息进行分类。此外，当对患者消息的响应更快时，患者在 5 天治疗窗口内获得抗病毒药物处方的可能性更大。尽管需要进一步分析对临床结果的影响，但这些发现代表了将 NLP 算法集成到临床护理中的一种可能应用。
