文献检索文档翻译深度研究
Suppr Zotero 插件Zotero 插件
邀请有礼套餐&价格历史记录

新学期,新优惠

限时优惠:9月1日-9月22日

30天高级会员仅需29元

1天体验卡首发特惠仅需5.99元

了解详情
不再提醒
插件&应用
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
高级版
套餐订阅购买积分包
AI 工具
文献检索文档翻译深度研究
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2025

利用大型语言模型从原始和去识别的医疗记录中提取复杂的健康社会决定因素摘要:开发和验证研究。

Using Large Language Models to Abstract Complex Social Determinants of Health From Original and Deidentified Medical Notes: Development and Validation Study.

机构信息

Institute for Systems Biology, Seattle, WA, United States.

Providence Health & Services, Renton, WA, United States.

出版信息

J Med Internet Res. 2024 Nov 19;26:e63445. doi: 10.2196/63445.


DOI:10.2196/63445
PMID:39561354
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC11615547/
Abstract

BACKGROUND: Social determinants of health (SDoH) such as housing insecurity are known to be intricately linked to patients' health status. More efficient methods for abstracting structured data on SDoH can help accelerate the inclusion of exposome variables in biomedical research and support health care systems in identifying patients who could benefit from proactive outreach. Large language models (LLMs) developed from Generative Pre-trained Transformers (GPTs) have shown potential for performing complex abstraction tasks on unstructured clinical notes. OBJECTIVE: Here, we assess the performance of GPTs on identifying temporal aspects of housing insecurity and compare results between both original and deidentified notes. METHODS: We compared the ability of GPT-3.5 and GPT-4 to identify instances of both current and past housing instability, as well as general housing status, from 25,217 notes from 795 pregnant women. Results were compared with manual abstraction, a named entity recognition model, and regular expressions. RESULTS: Compared with GPT-3.5 and the named entity recognition model, GPT-4 had the highest performance and had a much higher recall (0.924) than human abstractors (0.702) in identifying patients experiencing current or past housing instability, although precision was lower (0.850) compared with human abstractors (0.971). GPT-4's precision improved slightly (0.936 original, 0.939 deidentified) on deidentified versions of the same notes, while recall dropped (0.781 original, 0.704 deidentified). CONCLUSIONS: This work demonstrates that while manual abstraction is likely to yield slightly more accurate results overall, LLMs can provide a scalable, cost-effective solution with the advantage of greater recall. This could support semiautomated abstraction, but given the potential risk for harm, human review would be essential before using results for any patient engagement or care decisions. Furthermore, recall was lower when notes were deidentified prior to LLM abstraction.

摘要

背景:健康的社会决定因素(如住房无保障)与患者的健康状况密切相关,这是众所周知的。更有效的方法来提取关于社会决定因素的结构化数据,可以帮助加速外显子组变量纳入生物医学研究,并支持医疗保健系统识别可能受益于主动外展的患者。基于生成式预训练转换器(Generative Pre-trained Transformers,GPTs)开发的大型语言模型(Large language models,LLMs)已显示出在非结构化临床记录上执行复杂抽象任务的潜力。

目的:在这里,我们评估 GPT 在识别住房无保障的时间方面的性能,并比较原始和去识别记录之间的结果。

方法:我们比较了 GPT-3.5 和 GPT-4 从 795 名孕妇的 25217 份记录中识别当前和过去住房不稳定以及一般住房状况实例的能力。结果与人工抽象、命名实体识别模型和正则表达式进行了比较。

结果:与 GPT-3.5 和命名实体识别模型相比,GPT-4 在识别当前或过去住房不稳定的患者方面具有最高的性能,召回率(0.924)明显高于人工抽象者(0.702),尽管精度较低(0.850)。在相同记录的去识别版本上,GPT-4 的精度略有提高(0.936 原始,0.939 去识别),而召回率下降(0.781 原始,0.704 去识别)。

结论:这项工作表明,虽然人工抽象可能总体上产生更准确的结果,但 LLM 可以提供一种可扩展、具有成本效益的解决方案,具有更高的召回率优势。这可以支持半自动抽象,但考虑到潜在的伤害风险,在将结果用于任何患者参与或护理决策之前,人工审查是必不可少的。此外,在对 LLM 抽象之前对记录进行去识别时,召回率较低。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/3ab0/11615547/c1813107702c/jmir_v26i1e63445_fig1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/3ab0/11615547/c1813107702c/jmir_v26i1e63445_fig1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/3ab0/11615547/c1813107702c/jmir_v26i1e63445_fig1.jpg

相似文献

[1]
Using Large Language Models to Abstract Complex Social Determinants of Health From Original and Deidentified Medical Notes: Development and Validation Study.

J Med Internet Res. 2024-11-19

[2]
Using Large Language Models to Annotate Complex Cases of Social Determinants of Health in Longitudinal Clinical Records.

medRxiv. 2024-4-27

[3]
Validation of a Zero-shot Learning Natural Language Processing Tool to Facilitate Data Abstraction for Urologic Research.

Eur Urol Focus. 2024-3

[4]
Measuring the Value of a Practical Text Mining Approach to Identify Patients With Housing Issues in the Free-Text Notes in Electronic Health Record: Findings of a Retrospective Cohort Study.

Front Public Health. 2021

[5]
Quality of Answers of Generative Large Language Models Versus Peer Users for Interpreting Laboratory Test Results for Lay Patients: Evaluation Study.

J Med Internet Res. 2024-4-17

[6]
Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models.

Acad Emerg Med. 2024-6

[7]
Scalable information extraction from free text electronic health records using large language models.

BMC Med Res Methodol. 2025-1-28

[8]
A large language model-based generative natural language processing framework fine-tuned on clinical notes accurately extracts headache frequency from electronic health records.

Headache. 2024-4

[9]
Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing.

J Biomed Inform. 2022-3

[10]
Natural language processing to identify social determinants of health in Alzheimer's disease and related dementia from electronic health records.

Health Serv Res. 2023-12

引用本文的文献

[1]
Leveraging large language models for the deidentification and temporal normalization of sensitive health information in electronic health records.

NPJ Digit Med. 2025-8-13

[2]
Unveiling social determinants of health impact on adverse pregnancy outcomes through natural language processing.

Sci Rep. 2025-8-9

[3]
Extracting Multifaceted Characteristics of Patients With Chronic Disease Comorbidity: Framework Development Using Large Language Models.

JMIR Med Inform. 2025-5-15

本文引用的文献

[1]
Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges.

Proc Mach Learn Res. 2024-6

[2]
Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions.

Cell Rep Med. 2024-1-16

[3]
Large language models to identify social determinants of health in electronic health records.

NPJ Digit Med. 2024-1-11

[4]
Zero-shot interpretable phenotyping of postpartum hemorrhage using large language models.

NPJ Digit Med. 2023-11-30

[5]
Effect of COVID-19 vaccination and booster on maternal-fetal outcomes: a retrospective cohort study.

Lancet Digit Health. 2023-9

[6]
Trends, Characteristics, and Maternal Morbidity Associated With Unhoused Status in Pregnancy.

JAMA Netw Open. 2023-7-3

[7]
Large language models encode clinical knowledge.

Nature. 2023-8

[8]
Real-world integration of the protocol for responding to and assessing patients' assets, risks, and experiences tool to assess social determinants of health in the electronic medical record at an academic medical center.

Digit Health. 2023-5-22

[9]
Leveraging natural language processing to augment structured social determinants of health data in the electronic health record.

J Am Med Inform Assoc. 2023-7-19

[10]
A large language model for electronic health records.

NPJ Digit Med. 2022-12-26

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

推荐工具

医学文档翻译智能文献检索