Data Science, OM1 Inc, Boston, Massachusetts, USA.
Research, OM1 Inc, Boston, Massachusetts, USA
RMD Open. 2021 May;7(2). doi: 10.1136/rmdopen-2021-001586.
Use of the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) in routine clinical practice is inconsistent, and availability of clinician-recorded SLEDAI scores in real-world datasets is limited. This study aimed to validate a machine learning model to estimate SLEDAI score categories using clinical notes and to apply the model to a large, real-world dataset to generate estimated score categories for use in future research studies.
A machine learning model was developed to estimate an individual patient's SLEDAI score category (no activity, mild activity, moderate activity or high/very high activity) for a specific encounter date using clinical notes. A training cohort of 3504 encounters and a separate validation cohort of 1576 encounters were created from the OM1 SLE Registry. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), calculated using a binarised version of the outcome that sets the positive class to be those records with clinician-recorded SLEDAI scores >5 and the negative class to be records with scores ≤5. Model performance was evaluated by categorising the scores into the four disease activity categories and by calculating the Spearman's R value and Pearson's R value.
The AUC for the two categories was 0.93 for the development cohort and 0.91 for the validation cohort. The model had a Spearman's R value of 0.7 and a Pearson's R value of 0.7 when calculated using the four disease activity categories.
The model performs well when estimating SLEDAI score categories using unstructured clinical notes.
在常规临床实践中,红斑狼疮疾病活动指数(SLEDAI)的使用并不一致,并且在真实世界数据集中可用的临床医生记录的 SLEDAI 评分有限。本研究旨在验证一种机器学习模型,该模型使用临床记录来估计 SLEDAI 评分类别,并将该模型应用于大型真实世界数据集,以生成用于未来研究的估计评分类别。
开发了一种机器学习模型,用于使用临床记录来估计特定就诊日期个体患者的 SLEDAI 评分类别(无活动、轻度活动、中度活动或高/极高活动)。从 OM1 SLE 注册处创建了一个包含 3504 次就诊的训练队列和一个包含 1576 次就诊的单独验证队列。使用接受者操作特征曲线下的面积(AUC)评估模型性能,该面积是使用对结果进行二值化的方法计算得出的,将阳性类设置为临床记录的 SLEDAI 评分>5 的记录,将阴性类设置为评分≤5 的记录。通过将评分分为四个疾病活动类别,并计算 Spearman's R 值和 Pearson's R 值来评估模型性能。
开发队列的 AUC 为 0.93,验证队列的 AUC 为 0.91。当使用四个疾病活动类别进行计算时,该模型的 Spearman's R 值为 0.7,Pearson's R 值为 0.7。
该模型在使用非结构化临床记录估计 SLEDAI 评分类别时表现良好。