Arko Iv Leopold, Hudelson Carly, Kumar Jaya, Badlani Sameer, Stoffel Michelle, Markowitz Rebecca, Melton Genevieve B
University of Minnesota, Minneapolis, MN, USA.
M Health Fairview Systems, Minneapolis, MN, USA.
Stud Health Technol Inform. 2025 Aug 7;329:337-341. doi: 10.3233/SHTI250857.
Clinical documentation is a leading cause of clinician burnout. Artificial intelligence (AI) scribes use ambient generative AI to create text summaries of clinical encounters. While several commercial AI scribe tools are available, there are limited comparisons of the performance of these tools. This study evaluated the quality and accuracy of notes generated by four commercially available AI scribes using two simulated internal medicine and surgical patient encounters. AI-generated notes were compared to transcripts, with errors quantified and quality assessed using the Physician Documentation Quality Instrument (PDQI), evaluating succinctness, usefulness, and comprehensiveness. Omission errors constituted 71% of the total errors, addition errors made up 19.4%, and incorrect facts accounted for 6.5% of the errors identified among vendors. Vendor A had the lowest error rate (12.2%) and balanced note length and quality. Vendor C, with the highest omission error rate (24.4%), produced the shortest notes, while Vendor B generated the longest notes with the highest addition error rate (7.3%). PDQI-9 scores ranged from 35.5 to 39.5, with Vendor C & D achieving the highest score. These findings emphasize the need for ongoing AI tool oversight and highlight future research opportunities, including the impact of AI scribes on communication, safety, and documentation quality.
临床文档记录是导致临床医生职业倦怠的主要原因。人工智能（AI）抄写员使用环境生成式人工智能来创建临床会诊的文本摘要。虽然有几种商业AI抄写员工具可供使用，但对这些工具性能的比较却很有限。本研究使用两次模拟的内科和外科患者会诊评估了四种商用AI抄写员生成的记录的质量和准确性。将AI生成的记录与抄本进行比较，使用医生文档质量工具（PDQI）对错误进行量化并评估质量，评估简洁性、有用性和全面性。遗漏错误占总错误的71%，添加错误占19.4%，错误事实占供应商中识别出的错误的6.5%。供应商A的错误率最低（12.2%），且记录长度和质量平衡。供应商C的遗漏错误率最高（24.4%），生成的记录最短，而供应商B生成的记录最长，添加错误率最高（7.3%）。PDQI-9评分范围为35.5至39.5，供应商C和D得分最高。这些发现强调了对AI工具进行持续监督的必要性，并突出了未来的研究机会，包括AI抄写员对沟通、安全和文档质量的影响。
