Yang Jie, So Jonathan, Zhang Hao, Jones Simon, Connolly Denise M, Golding Claudia, Griffes Esmelin, Szerencsy Adam C, Wu Tzer Jason, Aphinyanaphongs Yindalon, Major Vincent J
Department of Health Informatics, NYU Langone Health, New York, NY 10016, United States.
Department of Population Health, NYU Grossman School of Medicine, New York, NY 10016, United States.
JAMIA Open. 2024 Aug 16;7(3):ooae078. doi: 10.1093/jamiaopen/ooae078. eCollection 2024 Oct.
Accelerating demand for patient messaging has impacted the practice of many providers. Messages are not recommended for urgent medical issues, but some do require rapid attention. This presents an opportunity for artificial intelligence (AI) methods to prioritize review of messages. Our study aimed to highlight some patient portal messages for prioritized review using a custom AI system integrated into the electronic health record (EHR).
We developed a Bidirectional Encoder Representations from Transformers (BERT)-based large language model using 40 132 patient-sent messages to identify patterns involving high acuity topics that warrant an immediate callback. The model was then implemented into 2 shared pools of patient messages managed by dozens of registered nurses. A primary outcome, such as the time before messages were read, was evaluated with a difference-in-difference methodology.
Model validation on an expert-reviewed dataset ( = 7260) yielded very promising performance (C-statistic = 97%, average-precision = 72%). A binarized output (precision = 67%, sensitivity = 63%) was integrated into the EHR for 2 years. In a pre-post analysis ( = 396 466), an improvement exceeding the trend was observed in the time high-scoring messages sit unread (21 minutes, 63 vs 42 for messages sent outside business hours).
Our work shows great promise in improving care when AI is aligned with human workflow. Future work involves audience expansion, aiding users with suggested actions, and drafting responses.
Many patients utilize patient portal messages, and while most messages are routine, a small fraction describe alarming symptoms. Our AI-based workflow shortens the turnaround time to get a trained clinician to review these messages to provide safer, higher-quality care.
对患者信息的需求加速，影响了许多医疗服务提供者的业务。不建议通过信息处理紧急医疗问题，但有些问题确实需要迅速关注。这为人工智能（AI）方法对信息审查进行优先级排序提供了机会。我们的研究旨在使用集成到电子健康记录（EHR）中的定制AI系统，突出一些需要优先审查的患者门户信息。
我们使用40132条患者发送的信息开发了一种基于双向编码器表征从变换器（BERT）的大语言模型，以识别涉及需要立即回电的高 acuity 主题的模式。然后将该模型应用于由数十名注册护士管理的2个共享患者信息池。通过差异-in-差异方法评估主要结果，例如信息被阅读之前的时间。
在经过专家审查的数据集（=7260）上进行的模型验证产生了非常有前景的性能（C统计量=97%，平均精度=72%）。一个二值化输出（精度=67%，灵敏度=63%）被集成到EHR中两年时间。在一项前后分析（=396466）中，观察到高分信息未被阅读的时间有超过趋势的改善（21分钟，非工作时间发送的信息从63分钟降至42分钟）。
我们的工作表明，当AI与人类工作流程相结合时，在改善护理方面具有很大潜力。未来的工作包括扩大受众群体、通过建议行动帮助用户以及起草回复。
许多患者使用患者门户信息，虽然大多数信息是常规的，但一小部分描述了令人担忧的症状。我们基于AI的工作流程缩短了让训练有素的临床医生审查这些信息以提供更安全、更高质量护理的周转时间。
