Jaiswal Aman, Katz Alan, Nesca Marcello, Milios Evangelos
Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada.
Manitoba Centre for Health Policy, Department of Community Health Sciences, University of Manitoba, Winnipeg, MB, Canada.
JMIR Med Inform. 2023 Aug 9;11:e45105. doi: 10.2196/45105.
Lower back pain is a common weakening condition that affects a large population. It is a leading cause of disability and lost productivity, and the associated medical costs and lost wages place a substantial burden on individuals and society. Recent advances in artificial intelligence and natural language processing have opened new opportunities for the identification and management of risk factors for lower back pain. In this paper, we propose and train a deep learning model on a data set of clinical notes that have been annotated with relevant risk factors, and we evaluate the model's performance in identifying risk factors in new clinical notes.
The primary objective is to develop a novel deep learning approach to detect risk factors for underlying disease in patients presenting with lower back pain in clinical encounter notes. The secondary objective is to propose solutions to potential challenges of using deep learning and natural language processing techniques for identifying risk factors in electronic medical record free text and make practical recommendations for future research in this area.
We manually annotated clinical notes for the presence of six risk factors for severe underlying disease in patients presenting with lower back pain. Data were highly imbalanced, with only 12% (n=296) of the annotated notes having at least one risk factor. To address imbalanced data, a combination of semantic textual similarity and regular expressions was used to further capture notes for annotation. Further analysis was conducted to study the impact of downsampling, binary formulation of multi-label classification, and unsupervised pretraining on classification performance.
Of 2749 labeled clinical notes, 347 exhibited at least one risk factor, while 2402 exhibited none. The initial analysis shows that downsampling the training set to equalize the ratio of clinical notes with and without risk factors improved the macro-area under the receiver operating characteristic curve (AUROC) by 2%. The Bidirectional Encoder Representations from Transformers (BERT) model improved the macro-AUROC by 15% over the traditional machine learning baseline. In experiment 2, the proposed BERT-convolutional neural network (CNN) model for longer texts improved (4% macro-AUROC) over the BERT baseline, and the multitask models are more stable for minority classes. In experiment 3, domain adaptation of BERTCNN using masked language modeling improved the macro-AUROC by 2%.
Primary care clinical notes are likely to require manipulation to perform meaningful free-text analysis. The application of BERT models for multi-label classification on downsampled annotated clinical notes is useful in detecting risk factors suggesting an indication for imaging for patients with lower back pain.
下背痛是一种常见的使人身体衰弱的病症，影响着大量人群。它是导致残疾和生产力丧失的主要原因，相关的医疗费用和工资损失给个人和社会带来了沉重负担。人工智能和自然语言处理的最新进展为下背痛风险因素的识别和管理带来了新机遇。在本文中，我们在已标注相关风险因素的临床记录数据集上提出并训练了一个深度学习模型，并评估该模型在识别新临床记录中的风险因素方面的性能。
主要目标是开发一种新颖的深度学习方法，以在临床会诊记录中检测下背痛患者潜在疾病的风险因素。次要目标是针对使用深度学习和自然语言处理技术在电子病历自由文本中识别风险因素的潜在挑战提出解决方案，并为该领域的未来研究提出实用建议。
我们手动标注了下背痛患者严重潜在疾病六个风险因素的临床记录。数据高度不平衡，只有12%（n = 296）的标注记录有至少一个风险因素。为解决数据不平衡问题，使用语义文本相似度和正则表达式的组合进一步获取用于标注的记录。进行了进一步分析，以研究下采样、多标签分类的二元化以及无监督预训练对分类性能的影响。
在2749条标注的临床记录中，347条显示至少一个风险因素，而2402条未显示任何风险因素。初步分析表明，对训练集进行下采样以均衡有和没有风险因素的临床记录比例，使接收器操作特征曲线（AUROC）下的宏面积提高了2%。与传统机器学习基线相比，来自变换器的双向编码器表示（BERT）模型使宏AUROC提高了15%。在实验2中，针对较长文本提出的BERT - 卷积神经网络（CNN）模型比BERT基线有所改进（宏AUROC提高4%），并且多任务模型对少数类更稳定。在实验3中，使用掩码语言建模对BERTCNN进行领域适应使宏AUROC提高了2%。
基层医疗临床记录可能需要进行处理以进行有意义的自由文本分析。在经过下采样的标注临床记录上应用BERT模型进行多标签分类，有助于检测提示下背痛患者进行影像学检查指征的风险因素。
