Proctor Stephon, Lawton Greg, Sinha Shikha
Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
Department of Biomedical and Health Informatics, University of Pennsylvania, Philadelphia, Pennsylvania.
Appl Clin Inform. 2025 Aug;16(4):747-752. doi: 10.1055/a-2576-0579. Epub 2025 Apr 8.
This study aims to evaluate the impact of using a large language model (LLM) for generating draft responses to patient messages in the electronic health record (EHR) system on clinicians and support staff workload and efficiency.We partnered with Epic Systems to implement OpenAI's ChatGPT 4.0 for responding to patient messages. A pilot study was conducted from August 2023 to July 2024 across 13 ambulatory specialties involving 323 participants, including clinicians and support staff. Data on draft utilization rates and message response times were collected and analyzed using statistical methods.The overall mean generated draft utilization rate was 38%, with significant differences by role and specialty. Clinicians had a higher utilization rate (43%) than scheduling staff (33%). Draft message usage significantly reduced all users' message response time (13 seconds on average). Support staff experienced a more substantial and statistically significant time saving (23 seconds) compared to negligible time savings seen by clinicians (3 seconds). Variability in utilization rates and time savings was observed across different specialties.Implementing LLMs for drafting patient message replies can reduce response times and alleviate message burden. However, the effectiveness of artificial intelligence (AI)-generated draft responses varies by clinical role and specialty, indicating the need for tailored implementations. Further investigation into this variability, and development and personalization of AI tools are recommended to maximize their utility and ensure safe and effective use in diverse clinical contexts.
本研究旨在评估在电子健康记录(EHR)系统中使用大语言模型(LLM)为患者信息生成回复草稿对临床医生和支持人员工作量及效率的影响。我们与Epic Systems合作,采用OpenAI的ChatGPT 4.0来回复患者信息。2023年8月至2024年7月开展了一项试点研究,涉及13个门诊专科的323名参与者,包括临床医生和支持人员。收集了草稿利用率和信息回复时间的数据,并使用统计方法进行分析。总体平均生成草稿利用率为38%,不同角色和专科存在显著差异。临床医生的利用率(43%)高于排班人员(33%)。草稿信息的使用显著缩短了所有用户的信息回复时间(平均13秒)。与临床医生可忽略不计的时间节省(3秒)相比,支持人员节省的时间更多且具有统计学意义(23秒)。不同专科的利用率和时间节省情况存在差异。使用大语言模型起草患者信息回复可以缩短回复时间并减轻信息负担。然而,人工智能(AI)生成的回复草稿的有效性因临床角色和专科而异,这表明需要进行量身定制的实施。建议进一步研究这种差异,并开发和个性化人工智能工具,以最大限度地发挥其效用,并确保在不同临床环境中安全有效地使用。