文献检索文档翻译深度研究
Suppr Zotero 插件Zotero 插件
邀请有礼套餐&价格历史记录

新学期,新优惠

限时优惠:9月1日-9月22日

30天高级会员仅需29元

1天体验卡首发特惠仅需5.99元

了解详情
不再提醒
插件&应用
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
高级版
套餐订阅购买积分包
AI 工具
文献检索文档翻译深度研究
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2025

对使用大语言模型进行患者对话摘要的综合定性分析,该分析应用于嘈杂、非正式、非英语的真实世界数据。

A comprehensive qualitative analysis of patient dialogue summarization using large language models applied to noisy, informal, non-English real-world data.

作者信息

Ferreira Anderson A, Rocha Leonardo, Cunha Washington, Machado Ana Cláudia, Campos João Marcos, Jallais Gabriel, Viana Adriana C F, Tuler Elisa, Araújo Iago, Macul Víctor, Souza Neto Olívio, de Souza Júnior Antônio Pereira, de Pinho Souza Giordano, Pallone Joice Marques, Dumbá Soares Mariana Aparecida, Santos Welton Augusto, Gonçalves Marcos André

机构信息

Computer Science Department, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil.

Computer Science Department, Universidade Federal de Ouro Preto, Ouro Preto, Minas Gerais, Brazil.

出版信息

Sci Rep. 2025 Aug 27;15(1):31660. doi: 10.1038/s41598-025-13560-9.


DOI:10.1038/s41598-025-13560-9
PMID:40866405
Abstract

This study evaluates the ability of Large Language Models (LLMs) to summarize real-world dialogues between patients and the healthcare team of an e-health company that provides digital healthcare services, primarily communicating via WhatsApp. The team needs quick access to patient information to deliver accurate and personalized responses. Summarizing past messages is the approach examined here, aiming for concise, non-redundant, and truthful summaries that capture the main dialogue characteristics despite facing (real-world) noisy and informal content in an under-represented language - Portuguese. To do so, we collected an anonymized Portuguese dataset of WhatsApp messages exchanged between patients and the healthcare team. Dialogue quality was assessed for size, readability, and correctness before generating summaries with LLaMA3 and Qwen2 using specific prompts. Volunteers evaluated these summaries on coverage, relevance, redundancy, and veracity using a 5-point Likert scale. Our qualitative and quantitative experimental results indicate that LLMs can produce effective summaries of dialogues between patients and healthcare teams, even when faced with low-quality data in an underrepresented language. This is a surprising result due to the challenging scenario. Among the tested LLMs, LLaMA3 demonstrated a slight edge over QWen2 in coverage and veracity among the evaluated methods. Our results demonstrate a potential to build real-world practical services to assist healthcare professionals in responding to patient messages with agility, clarity, and cohesion, enhancing both communication efficiency and patient satisfaction. Ultimately, the advocated approach could significantly improve the landscape of online healthcare communication, particularly in resource-constrained settings like Brazil, where access to primary care is limited.

摘要

本研究评估了大语言模型(LLMs)总结患者与一家提供数字医疗服务的电子医疗公司的医疗团队之间真实对话的能力,该公司主要通过WhatsApp进行沟通。该团队需要快速获取患者信息,以便提供准确且个性化的回复。总结过往信息是此处所研究的方法,旨在生成简洁、无冗余且真实的总结,尽管面对(现实世界中)以葡萄牙语这种代表性不足的语言呈现的嘈杂且非正式的内容,仍能捕捉主要对话特征。为此,我们收集了患者与医疗团队之间交换的WhatsApp消息的匿名葡萄牙语数据集。在使用特定提示词通过LLaMA3和Qwen2生成总结之前,对对话质量进行了大小、可读性和正确性方面的评估。志愿者使用5点李克特量表对这些总结在覆盖范围、相关性、冗余度和真实性方面进行了评估。我们的定性和定量实验结果表明,即使面对代表性不足的语言中的低质量数据,大语言模型也能够生成患者与医疗团队之间对话的有效总结。鉴于这种具有挑战性的场景,这是一个令人惊讶的结果。在所测试的大语言模型中,在评估方法中,LLaMA3在覆盖范围和真实性方面比QWen2略胜一筹。我们的结果表明,有潜力构建现实世界中的实用服务,以帮助医疗专业人员灵活、清晰且连贯地回复患者消息,提高沟通效率并提升患者满意度。最终,所倡导的方法可以显著改善在线医疗沟通的局面,特别是在像巴西这样资源有限的环境中,那里获得初级医疗服务的机会有限。

相似文献

[1]
A comprehensive qualitative analysis of patient dialogue summarization using large language models applied to noisy, informal, non-English real-world data.

Sci Rep. 2025-8-27

[2]
Prescription of Controlled Substances: Benefits and Risks

2025-1

[3]
Performance of ChatGPT-4o and Four Open-Source Large Language Models in Generating Diagnoses Based on China's Rare Disease Catalog: Comparative Study.

J Med Internet Res. 2025-6-18

[4]
Parents' and informal caregivers' views and experiences of communication about routine childhood vaccination: a synthesis of qualitative evidence.

Cochrane Database Syst Rev. 2017-2-7

[5]
Healthcare workers' informal uses of mobile phones and other mobile devices to support their work: a qualitative evidence synthesis.

Cochrane Database Syst Rev. 2024-8-27

[6]
Improving Large Language Models' Summarization Accuracy by Adding Highlights to Discharge Notes: Comparative Evaluation.

JMIR Med Inform. 2025-7-24

[7]
Accreditation through the eyes of nurse managers: an infinite staircase or a phenomenon that evaporates like water.

J Health Organ Manag. 2025-6-30

[8]
Neonatal Nurses' Understanding of the Factors That Enhance and Hinder Early Communication Between Preterm Infants and Their Parents: A Narrative Inquiry Study.

Int J Lang Commun Disord. 2025

[9]
A dataset and benchmark for hospital course summarization with adapted large language models.

J Am Med Inform Assoc. 2025-3-1

[10]
Health professionals' experience of teamwork education in acute hospital settings: a systematic review of qualitative literature.

JBI Database System Rev Implement Rep. 2016-4

本文引用的文献

[1]
Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review.

J Med Internet Res. 2024-11-7

[2]
Large language models in health care: Development, applications, and challenges.

Health Care Sci. 2023-7-24

[3]
Leveraging large language models for generating responses to patient messages-a subjective analysis.

J Am Med Inform Assoc. 2024-5-20

[4]
Patient Information Summarization in Clinical Settings: Scoping Review.

JMIR Med Inform. 2023-11-28

[5]
ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations.

Front Artif Intell. 2023-5-4

[6]
The impact of health information technology on patient safety.

Saudi Med J. 2017-12

[7]
Association between expansion of primary healthcare and racial inequalities in mortality amenable to primary care in Brazil: A national longitudinal analysis.

PLoS Med. 2017-5-30

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

推荐工具

医学文档翻译智能文献检索