School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China.
School of Biomedical Engineering (Suzhou) (T.E.K.,Y.C., H.S.), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, 230026, China; Department of Medical Imaging (T.E.K.,Y.C., H.S., J.Z., X.Y.), Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China; Hefei National Lab for Physical Sciences at the Microscale and Centres for Biomedical Engineering (B.A.N.), University of Science and Technology of China, Hefei, 230026, China; EasySignal Group, Department of Automation (P.M.), Tsinghua University, Beijing 100084, China; Department of Biomedical Engineering (E.O.O.), Shenzhen University, Shenzhen, 518060, China; Jinhua Laboratory (X.Y.), Foshan, 528000, China.
Acad Radiol. 2021 Nov;28(11):1507-1523. doi: 10.1016/j.acra.2021.08.008. Epub 2021 Sep 17.
To perform a meta-analysis to compare the diagnostic test accuracy (DTA) of deep learning (DL) in detecting coronavirus disease 2019 (COVID-19), and to investigate how network architecture and type of datasets affect DL performance.
We searched PubMed, Web of Science and Inspec from January 1, 2020, to December 3, 2020, for retrospective and prospective studies on deep learning detection with at least reported sensitivity and specificity. Pooled DTA was obtained using random-effect models. Sub-group analysis between studies was also carried out for data source and network architectures.
The pooled sensitivity and specificity were 91% (95% confidence interval [CI]: 88%, 93%; I = 69%) and 92% (95% CI: 88%, 94%; I = 88%), respectively for 19 studies. The pooled AUC and diagnostic odds ratio (DOR) were 0.95 (95% CI: 0.88, 0.92) and 112.5 (95% CI: 57.7, 219.3; I = 90%) respectively. The overall accuracy, recall, F1-score, LR and LR are 89.5%, 89.5%, 89.7%, 23.13 and 0.13. Sub-group analysis shows that the sensitivity and DOR significantly vary with the type of network architectures and sources of data with low heterogeneity are (I = 0%) and (I = 18%) for ResNet architecture and single-source datasets, respectively.
The diagnosis of COVID-19 via deep learning has achieved incredible performance, and the source of datasets, as well as network architectures, strongly affect DL performance.
进行荟萃分析以比较深度学习(DL)检测 2019 年冠状病毒病(COVID-19)的诊断测试准确性(DTA),并探讨网络架构和数据集类型如何影响 DL 性能。
我们从 2020 年 1 月 1 日至 2020 年 12 月 3 日,在 PubMed、Web of Science 和 Inspec 上搜索了关于使用至少报告了敏感性和特异性的深度学习检测的回顾性和前瞻性研究。使用随机效应模型获得汇总 DTA。还对数据源和网络架构进行了研究间的亚组分析。
19 项研究的汇总敏感性和特异性分别为 91%(95%置信区间[CI]:88%,93%;I=69%)和 92%(95% CI:88%,94%;I=88%)。汇总 AUC 和诊断比值比(DOR)分别为 0.95(95% CI:0.88,0.92)和 112.5(95% CI:57.7,219.3;I=90%)。总体准确性、召回率、F1 评分、LR 和 LR 分别为 89.5%、89.5%、89.7%、23.13 和 0.13。亚组分析表明,敏感性和 DOR 随网络架构和数据源的类型显著变化,具有低异质性的分别为(I=0%)和(I=18%)的 ResNet 架构和单源数据集。
深度学习诊断 COVID-19 的性能令人难以置信,并且数据集的来源以及网络架构强烈影响 DL 性能。