College of Science and Engineering, University of Minnesota, Minneapolis, USA.
Department of Pharmaceutical Care and Health Systems, University of Minnesota, Minneapolis, USA.
BMC Med Inform Decis Mak. 2022 Jul 7;22(Suppl 1):88. doi: 10.1186/s12911-022-01819-4.
Since no effective therapies exist for Alzheimer's disease (AD), prevention has become more critical through lifestyle status changes and interventions. Analyzing electronic health records (EHRs) of patients with AD can help us better understand lifestyle's effect on AD. However, lifestyle information is typically stored in clinical narratives. Thus, the objective of the study was to compare different natural language processing (NLP) models on classifying the lifestyle statuses (e.g., physical activity and excessive diet) from clinical texts in English.
Based on the collected concept unique identifiers (CUIs) associated with the lifestyle status, we extracted all related EHRs for patients with AD from the Clinical Data Repository (CDR) of the University of Minnesota (UMN). We automatically generated labels for the training data by using a rule-based NLP algorithm. We conducted weak supervision for pre-trained Bidirectional Encoder Representations from Transformers (BERT) models and three traditional machine learning models as baseline models on the weakly labeled training corpus. These models include the BERT base model, PubMedBERT (abstracts + full text), PubMedBERT (only abstracts), Unified Medical Language System (UMLS) BERT, Bio BERT, Bio-clinical BERT, logistic regression, support vector machine, and random forest. The rule-based model used for weak supervision was tested on the GSC for comparison. We performed two case studies: physical activity and excessive diet, in order to validate the effectiveness of BERT models in classifying lifestyle status for all models were evaluated and compared on the developed Gold Standard Corpus (GSC) on the two case studies.
The UMLS BERT model achieved the best performance for classifying status of physical activity, with its precision, recall, and F-1 scores of 0.93, 0.93, and 0.92, respectively. Regarding classifying excessive diet, the Bio-clinical BERT model showed the best performance with precision, recall, and F-1 scores of 0.93, 0.93, and 0.93, respectively.
The proposed approach leveraging weak supervision could significantly increase the sample size, which is required for training the deep learning models. By comparing with the traditional machine learning models, the study also demonstrates the high performance of BERT models for classifying lifestyle status for Alzheimer's disease in clinical notes.
由于目前尚无治疗阿尔茨海默病（AD）的有效疗法，因此通过改变生活方式和干预措施来预防疾病变得更加重要。分析 AD 患者的电子健康记录（EHR）可以帮助我们更好地了解生活方式对 AD 的影响。然而，生活方式信息通常存储在临床叙述中。因此，本研究的目的是比较不同的自然语言处理（NLP）模型在从英语临床文本中分类生活方式状态（例如，身体活动和过度饮食）方面的性能。
基于与生活方式状态相关的收集概念唯一标识符（CUI），我们从明尼苏达大学（UMN）的临床数据存储库（CDR）中提取了所有 AD 患者的相关 EHR。我们使用基于规则的 NLP 算法自动为训练数据生成标签。我们对经过预训练的基于双向编码器表示的转换器（BERT）模型和三个传统机器学习模型（作为基线模型）进行了弱监督，这些模型包括 BERT 基础模型、PubMedBERT（摘要+全文）、PubMedBERT（仅摘要）、统一医学语言系统（UMLS）BERT、Bio BERT、Bio-clinical BERT、逻辑回归、支持向量机和随机森林。用于弱监督的基于规则的模型在 GSC 上进行了测试，以便进行比较。我们进行了两项案例研究：身体活动和过度饮食，以便验证 BERT 模型在分类生活方式状态方面的有效性，所有模型都在两项案例研究的开发的金标准语料库（GSC）上进行了评估和比较。
UMLS BERT 模型在分类身体活动状态方面表现最佳，其精度、召回率和 F1 分数分别为 0.93、0.93 和 0.92。关于分类过度饮食，Bio-clinical BERT 模型表现最佳，其精度、召回率和 F1 分数分别为 0.93、0.93 和 0.93。
该研究提出的利用弱监督的方法可以显著增加训练深度学习模型所需的样本量。通过与传统机器学习模型进行比较，该研究还证明了 BERT 模型在分类 AD 临床记录中的生活方式状态方面的高性能。
