Jin Haomiao, Chien Sandy, Meijer Erik, Khobragade Pranali, Lee Jinkook
Center for Economic and Social Research, University of Southern California, Los Angeles, CA, United States.
RAND Corporation, Santa Monica, CA, United States.
JMIR Ment Health. 2021 May 10;8(5):e27113. doi: 10.2196/27113.
The Harmonized Diagnostic Assessment of Dementia for the Longitudinal Aging Study in India (LASI-DAD) is the first and only nationally representative study on late-life cognition and dementia in India (n=4096). LASI-DAD obtained clinical consensus diagnosis of dementia for a subsample of 2528 respondents.
This study develops a machine learning model that uses data from the clinical consensus diagnosis in LASI-DAD to support the classification of dementia status.
Clinicians were presented with the extensive data collected from LASI-DAD, including sociodemographic information and health history of respondents, results from the screening tests of cognitive status, and information obtained from informant interviews. Based on the Clinical Dementia Rating (CDR) and using an online platform, clinicians individually evaluated each case and then reached a consensus diagnosis. A 2-step procedure was implemented to train several candidate machine learning models, which were evaluated using a separate test set for predictive accuracy measurement, including the area under receiver operating curve (AUROC), accuracy, sensitivity, specificity, precision, F1 score, and kappa statistic. The ultimate model was selected based on overall agreement as measured by kappa. We further examined the overall accuracy and agreement with the final consensus diagnoses between the selected machine learning model and individual clinicians who participated in the clinical consensus diagnostic process. Finally, we applied the selected model to a subgroup of LASI-DAD participants for whom the clinical consensus diagnosis was not obtained to predict their dementia status.
Among the 2528 individuals who received clinical consensus diagnosis, 192 (6.7% after adjusting for sampling weight) were diagnosed with dementia. All candidate machine learning models achieved outstanding discriminative ability, as indicated by AUROC >.90, and had similar accuracy and specificity (both around 0.95). The support vector machine model outperformed other models with the highest sensitivity (0.81), F1 score (0.72), and kappa (.70, indicating substantial agreement) and the second highest precision (0.65). As a result, the support vector machine was selected as the ultimate model. Further examination revealed that overall accuracy and agreement were similar between the selected model and individual clinicians. Application of the prediction model on 1568 individuals without clinical consensus diagnosis classified 127 individuals as living with dementia. After applying sampling weight, we can estimate the prevalence of dementia in the population as 7.4%.
The selected machine learning model has outstanding discriminative ability and substantial agreement with a clinical consensus diagnosis of dementia. The model can serve as a computer model of the clinical knowledge and experience encoded in the clinical consensus diagnostic process and has many potential applications, including predicting missed dementia diagnoses and serving as a clinical decision support tool or virtual rater to assist diagnosis of dementia.
印度纵向老龄化研究中的痴呆症统一诊断评估（LASI-DAD）是印度首个也是唯一一项关于晚年认知和痴呆症的具有全国代表性的研究（n = 4096）。LASI-DAD对2528名受访者的子样本进行了痴呆症的临床共识诊断。
本研究开发一种机器学习模型，该模型使用LASI-DAD临床共识诊断中的数据来支持痴呆症状态的分类。
向临床医生提供从LASI-DAD收集的大量数据，包括受访者的社会人口统计学信息和健康史、认知状态筛查测试结果以及从 informant 访谈中获得的信息。基于临床痴呆评定量表（CDR）并使用在线平台，临床医生分别评估每个病例，然后达成共识诊断。实施了一个两步程序来训练几个候选机器学习模型，并使用一个单独的测试集对其进行评估以测量预测准确性，包括受试者工作特征曲线下面积（AUROC）、准确性、敏感性、特异性、精确性、F1分数和kappa统计量。根据kappa测量的总体一致性选择最终模型。我们进一步检查了所选机器学习模型与参与临床共识诊断过程的个体临床医生之间的总体准确性以及与最终共识诊断的一致性。最后，我们将所选模型应用于LASI-DAD参与者中未获得临床共识诊断的一个亚组，以预测他们的痴呆症状态。
在接受临床共识诊断的2528人中，192人（调整抽样权重后为6.7%）被诊断为痴呆症。所有候选机器学习模型都具有出色的判别能力，AUROC>.90表明了这一点，并且具有相似的准确性和特异性（均约为0.95）。支持向量机模型在敏感性（0.81）、F1分数（0.72）和kappa（.70，表明高度一致性）方面表现优于其他模型，精确性第二高（0.65）。因此，支持向量机被选为最终模型。进一步检查发现所选模型与个体临床医生之间的总体准确性和一致性相似。将预测模型应用于1568名未获得临床共识诊断的个体，将127人分类为患有痴呆症。应用抽样权重后，我们可以估计人群中痴呆症的患病率为7.4%。
所选的机器学习模型具有出色的判别能力，并且与痴呆症的临床共识诊断高度一致。该模型可以作为临床共识诊断过程中编码的临床知识和经验的计算机模型，并且有许多潜在应用，包括预测漏诊的痴呆症诊断以及作为临床决策支持工具或虚拟评分器来辅助痴呆症诊断。
