Lee Scott H, Fox Shannon, Smith Raheem, Skrobarcek Kimberly A, Keyserling Harold, Phares Christina R, Lee Deborah, Posey Drew L
National Center for Emerging and Zoonotic Infectious Diseases, US Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America.
G2S Corporation, San Antonio, Texas, United States of America.
PLOS Digit Health. 2024 Sep 30;3(9):e0000612. doi: 10.1371/journal.pdig.0000612. eCollection 2024 Sep.
Immigrants and refugees seeking admission to the United States must first undergo an overseas medical exam, overseen by the US Centers for Disease Control and Prevention (CDC), during which all persons ≥15 years old receive a chest x-ray to look for signs of tuberculosis. Although individual screening sites often implement quality control (QC) programs to ensure radiographs are interpreted correctly, the CDC does not currently have a method for conducting similar QC reviews at scale. We obtained digitized chest radiographs collected as part of the overseas immigration medical exam. Using radiographs from applicants 15 years old and older, we trained deep learning models to perform three tasks: identifying abnormal radiographs; identifying abnormal radiographs suggestive of tuberculosis; and identifying the specific findings (e.g., cavities or infiltrates) in abnormal radiographs. We then evaluated the models on both internal and external testing datasets, focusing on two classes of performance metrics: individual-level metrics, like sensitivity and specificity, and sample-level metrics, like accuracy in predicting the prevalence of abnormal radiographs. A total of 152,012 images (one image per applicant; mean applicant age 39 years) were used for model training. On our internal test dataset, our models performed well both in identifying abnormalities suggestive of TB (area under the curve [AUC] of 0.97; 95% confidence interval [CI]: 0.95, 0.98) and in estimating sample-level counts of the same (-2% absolute percentage error; 95% CIC: -8%, 6%). On the external test datasets, our models performed similarly well in identifying both generic abnormalities (AUCs ranging from 0.89 to 0.92) and those suggestive of TB (AUCs from 0.94 to 0.99). This performance was consistent across metrics, including those based on thresholded class predictions, like sensitivity, specificity, and F1 score. Strong performance relative to high-quality radiological reference standards across a variety of datasets suggests our models may make reliable tools for supporting chest radiography QC activities at CDC.
寻求进入美国的移民和难民必须首先接受由美国疾病控制与预防中心（CDC）监督的海外体检，在此过程中，所有15岁及以上的人都要进行胸部X光检查，以查找结核病迹象。尽管各个筛查地点通常会实施质量控制（QC）程序以确保X光片得到正确解读，但CDC目前尚无大规模进行类似QC审查的方法。我们获取了作为海外移民体检一部分而收集的数字化胸部X光片。利用15岁及以上申请人的X光片，我们训练了深度学习模型以执行三项任务：识别异常X光片；识别提示结核病的异常X光片；以及识别异常X光片中的具体表现（如空洞或浸润）。然后，我们在内部和外部测试数据集上对模型进行了评估，重点关注两类性能指标：个体层面的指标，如敏感性和特异性；以及样本层面的指标，如预测异常X光片患病率的准确性。总共152,012张图像（每位申请人一张图像；申请人平均年龄39岁）用于模型训练。在我们的内部测试数据集上，我们的模型在识别提示结核病的异常情况（曲线下面积[AUC]为0.97；95%置信区间[CI]：0.95，0.98）以及估计相同情况的样本层面数量方面（绝对百分比误差为-2%；95% CIC：-8%，6%）表现良好。在外部测试数据集上，我们的模型在识别一般异常情况（AUC范围为0.89至0.92）和提示结核病的异常情况（AUC范围为0.94至0.99）方面表现同样出色。这种性能在各项指标中保持一致，包括基于阈值分类预测的指标，如敏感性、特异性和F1分数。在各种数据集上相对于高质量放射学参考标准的强劲表现表明，我们的模型可能成为支持CDC胸部X光检查QC活动的可靠工具。
