Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.
Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts.
JAMA Netw Open. 2021 Nov 1;4(11):e2135174. doi: 10.1001/jamanetworkopen.2021.35174.
Detecting cognitive decline earlier among older adults can facilitate enrollment in clinical trials and early interventions. Clinical notes in longitudinal electronic health records (EHRs) provide opportunities to detect cognitive decline earlier than it is noted in structured EHR fields as formal diagnoses.
To develop and validate a deep learning model to detect evidence of cognitive decline from clinical notes in the EHR.
DESIGN, SETTING, AND PARTICIPANTS: Notes documented 4 years preceding the initial mild cognitive impairment (MCI) diagnosis were extracted from Mass General Brigham's Enterprise Data Warehouse for patients aged 50 years or older and with initial MCI diagnosis during 2019. The study was conducted from March 1, 2020, to June 30, 2021. Sections of notes for cognitive decline were labeled manually and 2 reference data sets were created. Data set I contained a random sample of 4950 note sections filtered by a list of keywords related to cognitive functions and was used for model training and testing. Data set II contained 2000 randomly selected sections without keyword filtering for assessing whether the model performance was dependent on specific keywords.
A deep learning model and 4 baseline models were developed and their performance was compared using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC).
Data set I represented 1969 patients (1046 [53.1%] women; mean [SD] age, 76.0 [13.3] years). Data set II comprised 1161 patients (619 [53.3%] women; mean [SD] age, 76.5 [10.2] years). With some overlap of patients deleted, the unique population was 2166. Cognitive decline was noted in 1453 sections (29.4%) in data set I and 69 sections (3.45%) in data set II. Compared with the 4 baseline models, the deep learning model achieved the best performance in both data sets, with AUROC of 0.971 (95% CI, 0.967-0.976) and AUPRC of 0.933 (95% CI, 0.921-0.944) for data set I and AUROC of 0.997 (95% CI, 0.994-0.999) and AUPRC of 0.929 (95% CI, 0.870-0.969) for data set II.
In this diagnostic study, a deep learning model accurately detected cognitive decline from clinical notes preceding MCI diagnosis and had better performance than keyword-based search and other machine learning models. These results suggest that a deep learning model could be used for earlier detection of cognitive decline in the EHRs.
在老年人中更早地发现认知能力下降，可以促进临床试验和早期干预的参与。纵向电子健康记录（EHR）中的临床记录提供了比在结构化 EHR 字段中作为正式诊断更早地发现认知能力下降的机会。
开发和验证一种深度学习模型，以从 EHR 中的临床记录中检测认知能力下降的证据。
设计、设置和参与者：从马萨诸塞州综合医院的企业数据仓库中提取了 2019 年年龄在 50 岁或以上且初始 MCI 诊断期间记录的 4 年前的临床记录。该研究于 2020 年 3 月 1 日至 2021 年 6 月 30 日进行。手动标记认知下降部分的记录，并创建了 2 个参考数据集。数据集 I 包含通过与认知功能相关的关键字列表过滤的 4950 个记录部分的随机样本，用于模型训练和测试。数据集 II 包含 2000 个随机选择的无关键字过滤部分，用于评估模型性能是否取决于特定的关键字。
开发了一种深度学习模型和 4 个基线模型，并使用接收器工作特征曲线下的面积（AUROC）和精度召回曲线下的面积（AUPRC）比较了它们的性能。
数据集 I 代表了 1969 名患者（1046 名[53.1%]女性；平均[SD]年龄，76.0[13.3]岁）。数据集 II 包括 1161 名患者（619 名[53.3%]女性；平均[SD]年龄，76.5[10.2]岁）。删除一些重叠的患者后，独特的人群为 2166 人。在数据集 I 中，1453 个部分（29.4%）注意到认知能力下降，在数据集 II 中，69 个部分（3.45%）注意到认知能力下降。与 4 个基线模型相比，深度学习模型在两个数据集的表现都最好，数据集 I 的 AUROC 为 0.971（95%CI，0.967-0.976），AUPRC 为 0.933（95%CI，0.921-0.944），数据集 II 的 AUROC 为 0.997（95%CI，0.994-0.999），AUPRC 为 0.929（95%CI，0.870-0.969）。
在这项诊断研究中，深度学习模型准确地从 MCI 诊断前的临床记录中检测到认知能力下降，其性能优于基于关键字的搜索和其他机器学习模型。这些结果表明，深度学习模型可用于 EHR 中认知能力下降的早期检测。
