Department of Radiological Sciences, Oncology and Pathology, "Sapienza" University of Rome, I.C.O.T. Hospital, Via Franco Faggiana, 1668, 04100, Latina, Italy.
Department of Diagnostic Imaging and Interventional Radiology, Santa Maria Goretti Hospital, 04100, Latina, Italy.
Eur Radiol. 2021 Apr;31(4):1932-1940. doi: 10.1007/s00330-020-07273-y. Epub 2020 Sep 23.
To conduct a multireader validation study to evaluate the interobserver variability and the diagnostic accuracy for the lung involvement by COVID-19 of COVID-19 Reporting and Data System (CO-RADS) score.
This retrospective study included consecutive symptomatic patients who underwent chest CT and reverse transcriptase-polymerase chain reaction (RT-PCR) from March 2020 to May 2020 for suspected COVID-19. Twelve readers with different levels of expertise independently scored each CT using the CO-RADS scheme for detecting pulmonary involvement by COVID-19. Receiver operating characteristic (ROC) curves were computed to investigate diagnostic yield. Fleiss' kappa statistics was used to evaluate interreader agreement.
A total of 572 patients (mean age, 63 ± 20 [standard deviation]; 329 men; 142 patients with COVID-19 and 430 patients without COVID-19) were evaluated. There was a moderate agreement for CO-RADS rating among all readers (Fleiss' K = 0.43 [95% CI 0.42-0.44]) with a substantial agreement for CO-RADS 1 category (Fleiss' K = 0.61 [95% CI 0.60-0.62]) and moderate agreement for CO-RADS 5 category (Fleiss' K = 0.60 [95% CI 0.58-0.61]). ROC analysis showed the CO-RADS score ≥ 4 as the optimal threshold, with a cumulative area under the curve of 0.72 (95% CI 66-78%), sensitivity 61% (95% CI 52-69%), and specificity 81% (95% CI 77-84%).
CO-RADS showed high diagnostic accuracy and moderate interrater agreement across readers with different levels of expertise. Specificity is higher than previously thought and that could lead to reconsider the role of CT in this clinical setting.
• COVID-19 Reporting and Data System (CO-RADS) demonstrated a good diagnostic accuracy for lung involvement by COVID-19 with an average AUC of 0.72 (95% CI 67-75%). • When a threshold of ≥ 4 was used, sensitivity and specificity were 61% (95% CI 52-69%) and 81% (95% CI 76-84%), respectively. • There was an overall moderate agreement for CO-RADS rating across readers with different levels of expertise (Fleiss' K = 0.43 [95% CI 0.42-0.44]).
进行多读者验证研究，以评估 COVID-19 报告和数据系统（CO-RADS）评分对 COVID-19 肺部受累的观察者间变异性和诊断准确性。
本回顾性研究纳入了 2020 年 3 月至 2020 年 5 月间因疑似 COVID-19 而行胸部 CT 和逆转录-聚合酶链反应（RT-PCR）检查的连续有症状患者。12 名具有不同专业水平的读者分别使用 CO-RADS 方案对每个 CT 进行评分，以检测 COVID-19 的肺部受累。计算接收者操作特征（ROC）曲线以研究诊断效果。使用 Fleiss' kappa 统计量评估观察者间一致性。
共评估了 572 例患者（平均年龄 63 ± 20 [标准差]；329 名男性；142 例 COVID-19 患者和 430 例非 COVID-19 患者）。所有读者的 CO-RADS 评分存在中度一致性（Fleiss' K = 0.43 [95%CI 0.42-0.44]），CO-RADS 1 类具有显著一致性（Fleiss' K = 0.61 [95%CI 0.60-0.62]），CO-RADS 5 类具有中度一致性（Fleiss' K = 0.60 [95%CI 0.58-0.61]）。ROC 分析显示，CO-RADS 评分≥4 为最佳截断值，累积曲线下面积为 0.72（95%CI 66-78%），敏感性 61%（95%CI 52-69%），特异性 81%（95%CI 77-84%）。
CO-RADS 对 COVID-19 肺部受累具有较高的诊断准确性和中等观察者间一致性，在不同专业水平的读者中均如此。特异性高于先前的预期，这可能导致重新考虑 CT 在这一临床环境中的作用。
• COVID-19 报告和数据系统（CO-RADS）对 COVID-19 肺部受累的诊断准确性较好，平均 AUC 为 0.72（95%CI 67-75%）。
• 当使用≥4 的截断值时，敏感性和特异性分别为 61%（95%CI 52-69%）和 81%（95%CI 76-84%）。
• 在不同专业水平的读者中，CO-RADS 评分的总体一致性为中度（Fleiss' K = 0.43 [95%CI 0.42-0.44]）。
