利用电子健康记录的自然语言处理来衡量患者优先事项匹配护理的采用情况：模型的开发与验证

Measuring Adoption of Patient Priorities-Aligned Care Using Natural Language Processing of Electronic Health Records: Development and Validation of the Model.

作者信息

Razjouyan Javad, Freytag Jennifer, Dindo Lilian, Kiefer Lea, Odom Edward, Halaszynski Jaime, Silva Jennifer W, Naik Aanand D

机构信息

VA Health Services Research and Development Service, Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, TX, United States.

Department of Medicine, Baylor College of Medicine, Houston, TX, United States.

出版信息

JMIR Med Inform. 2021 Feb 19;9(2):e18756. doi: 10.2196/18756.

DOI:10.2196/18756

PMID:33605893

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC7935648/

Abstract

BACKGROUND

Patient Priorities Care (PPC) is a model of care that aligns health care recommendations with priorities of older adults who have multiple chronic conditions. Following identification of patient priorities, this information is documented in the patient's electronic health record (EHR).

OBJECTIVE

Our goal is to develop and validate a natural language processing (NLP) model that reliably documents when clinicians identify patient priorities (ie, values, outcome goals, and care preferences) within the EHR as a measure of PPC adoption.

METHODS

This is a retrospective analysis of unstructured National Veteran Health Administration EHR free-text notes using an NLP model. The data were sourced from 778 patient notes of 658 patients from encounters with 144 social workers in the primary care setting. Each patient's free-text clinical note was reviewed by 2 independent reviewers for the presence of PPC language such as priorities, values, and goals. We developed an NLP model that utilized statistical machine learning approaches. The performance of the NLP model in training and validation with 10-fold cross-validation is reported via accuracy, recall, and precision in comparison to the chart review.

RESULTS

Of 778 notes, 589 (75.7%) were identified as containing PPC language (kappa=0.82, P<.001). The NLP model in the training stage had an accuracy of 0.98 (95% CI 0.98-0.99), a recall of 0.98 (95% CI 0.98-0.99), and precision of 0.98 (95% CI 0.97-1.00). The NLP model in the validation stage had an accuracy of 0.92 (95% CI 0.90-0.94), recall of 0.84 (95% CI 0.79-0.89), and precision of 0.84 (95% CI 0.77-0.91). In contrast, an approach using simple search terms for PPC only had a precision of 0.757.

CONCLUSIONS

An automated NLP model can reliably measure with high precision, recall, and accuracy when clinicians document patient priorities as a key step in the adoption of PPC.

摘要

背景

患者优先护理（PPC）是一种将医疗保健建议与患有多种慢性病的老年人的优先事项相匹配的护理模式。在确定患者的优先事项后，这些信息会记录在患者的电子健康记录（EHR）中。

目的

我们的目标是开发并验证一种自然语言处理（NLP）模型，该模型能够可靠地记录临床医生在电子健康记录中确定患者优先事项（即价值观、结果目标和护理偏好）的时间，以此作为衡量PPC采用情况的指标。

方法

这是一项使用NLP模型对美国退伍军人健康管理局非结构化电子健康记录自由文本笔记进行的回顾性分析。数据来源于658名患者的778份患者笔记，这些笔记来自与144名初级保健环境中的社会工作者的会诊。由2名独立审阅者对每位患者的自由文本临床笔记进行审查，以确定是否存在如优先事项、价值观和目标等PPC相关语言。我们开发了一种利用统计机器学习方法的NLP模型。通过与图表审查相比的准确性、召回率和精确率，报告了NLP模型在10折交叉验证的训练和验证中的性能。

结果

在778份笔记中，589份（75.7%）被确定包含PPC相关语言（kappa = 0.82，P <.001）。训练阶段的NLP模型准确率为0.98（95%CI 0.98 - 0.99），召回率为0.98（95%CI 0.98 - 0.99），精确率为0.98（95%CI 0.97 - 1.00）。验证阶段的NLP模型准确率为0.92（95%CI 0.90 - 0.94），召回率为0.84（95%CI 0.79 - 0.89），精确率为0.84（95%CI 0.77 - 0.91）。相比之下，仅使用PPC简单搜索词的方法精确率仅为0.757。