Data Science, OM1 Inc, Boston, Massachusetts, USA.
Research, OM1 Inc, Boston, Massachusetts, USA
RMD Open. 2021 Nov;7(3). doi: 10.1136/rmdopen-2021-001781.
Disease activity measures, such as the Clinical Disease Activity Index (CDAI), are important tools for informing treatment decisions and monitoring patient outcomes in rheumatoid arthritis (RA). Yet, documentation of CDAI scores in electronic medical records and other real-world data sources is inconsistent, making it challenging to use these data for research. The purpose of this study was to validate a machine learning model to estimate CDAI scores for patients with RA using clinical notes.
A machine learning model was developed to estimate CDAI score values using clinical notes from a specific rheumatology visit. Data from the OM1 RA Registry were used to create a training cohort of 56 177 encounters and a separate validation cohort of 18 726 encounters, 11 985 of which passed a model-derived confidence filter; all included encounters had both a clinician-recorded CDAI score and a clinical note. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), positive predictive value (PPV) and negative predictive value (NPV), calculated using a binarised version of the outcome. The Spearman's R and Pearson's R values were also calculated.
The model had a PPV of 0.80, NPV of 0.84 and AUC of 0.88 when evaluating performance using the binarised version of the outcome. The model had a Spearman's R value of 0.72 and a Pearson's R value of 0.69 when evaluating performance using the continuous CDAI numeric scores.
A machine learning model estimates CDAI scores from clinical notes with good performance. Application of the model to real-world data sets may allow estimated CDAI scores to be used for research purposes.
疾病活动度指标，如临床疾病活动指数（CDAI），是为类风湿关节炎（RA）患者提供治疗决策和监测患者结局的重要工具。然而，电子病历和其他真实世界数据源中 CDAI 评分的记录不一致，使得这些数据难以用于研究。本研究的目的是验证一种使用临床记录估算 RA 患者 CDAI 评分的机器学习模型。
开发了一种机器学习模型，用于使用特定风湿病就诊的临床记录估算 CDAI 评分值。OM1 RA 登记处的数据用于创建一个 56177 次就诊的训练队列和一个 18726 次就诊的独立验证队列，其中 11985 次就诊通过了模型衍生的置信度筛选；所有纳入的就诊均有临床医生记录的 CDAI 评分和临床记录。使用接受者操作特征曲线下面积（AUC）、阳性预测值（PPV）和阴性预测值（NPV）评估模型性能，均使用二分结局计算。还计算了 Spearman's R 和 Pearson's R 值。
使用二分结局评估模型性能时，模型的 PPV 为 0.80，NPV 为 0.84，AUC 为 0.88。使用连续 CDAI 数值评分评估模型性能时，模型的 Spearman's R 值为 0.72，Pearson's R 值为 0.69。
机器学习模型可从临床记录中估算 CDAI 评分，性能良好。将该模型应用于真实世界数据集可能允许使用估算的 CDAI 评分进行研究。
