Department of Radiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China.
Institute of Advanced Research, Infervision Medical Technology Co. Ltd., Beijing, China.
Med Phys. 2022 Jun;49(6):3874-3885. doi: 10.1002/mp.15549. Epub 2022 Apr 18.
Artificial intelligence (AI) has been proved to be a highly efficient tool for COVID-19 diagnosis, but the large data size and heavy label force required for algorithm development and the poor generalizability of AI algorithms, to some extent, limit the application of AI technology in clinical practice. The aim of this study is to develop an AI algorithm with high robustness using limited chest CT data for COVID-19 discrimination.
A three dimensional algorithm that combined multi-instance learning with the LSTM architecture (3DMTM) was developed for differentiating COVID-19 from community acquired pneumonia (CAP) while logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), and a three dimensional convolutional neural network set for comparison. Totally, 515 patients with or without COVID-19 between December 2019 and March 2020 from five different hospitals were recruited and divided into relatively large (150 COVID-19 and 183 CAP cases) and relatively small datasets (17 COVID-19 and 35 CAP cases) for either training or validation and another independent dataset (37 COVID-19 and 93 CAP cases) for external test. Area under the receiver operating characteristic curve (AUC), sensitivity, specificity, precision, accuracy, F1 score, and G-mean were utilized for performance evaluation.
In the external test cohort, the relatively large data-based 3DMTM-LD achieved an AUC of 0.956 (95% confidence interval, 95% CI, 0.929∼0.982) with 86.2% and 98.0% for its sensitivity and specificity. 3DMTM-SD got an AUC of 0.937 (95% CI, 0.909∼0.965), while the AUC of 3DCM-SD decreased dramatically to 0.714 (95% CI, 0.649∼0.780) with training data reduction. KNN-MMSD, LR-MMSD, SVM-MMSD, and 3DCM-MMSD benefited significantly from the inclusion of clinical information while models trained with relatively large dataset got slight performance improvement in COVID-19 discrimination. 3DMTM, trained with either CT or multi-modal data, presented comparably excellent performance in COVID-19 discrimination.
The 3DMTM algorithm presented excellent robustness for COVID-19 discrimination with limited CT data. 3DMTM based on CT data performed comparably in COVID-19 discrimination with that trained with multi-modal information. Clinical information could improve the performance of KNN, LR, SVM, and 3DCM in COVID-19 discrimination, especially in the scenario with limited data for training.
人工智能（AI）已被证明是 COVID-19 诊断的高效工具，但算法开发需要大量的数据和繁重的标签，以及 AI 算法的通用性较差，在某种程度上限制了 AI 技术在临床实践中的应用。本研究旨在使用有限的胸部 CT 数据开发具有高稳健性的 AI 算法，以区分 COVID-19 和社区获得性肺炎（CAP）。
我们开发了一种将多实例学习与 LSTM 架构相结合的三维算法（3DMTM），用于区分 COVID-19 和 CAP，同时比较逻辑回归（LR）、k-最近邻（KNN）、支持向量机（SVM）和三维卷积神经网络。总共招募了 2019 年 12 月至 2020 年 3 月来自五家不同医院的 515 例有或无 COVID-19 的患者，将其分为相对较大（150 例 COVID-19 和 183 例 CAP）和相对较小数据集（17 例 COVID-19 和 35 例 CAP）进行训练或验证，以及另一个独立数据集（37 例 COVID-19 和 93 例 CAP）进行外部测试。使用受试者工作特征曲线下的面积（AUC）、灵敏度、特异性、精度、准确性、F1 评分和 G-均值来评估性能。
在外部测试队列中，基于较大数据的 3DMTM-LD 的 AUC 为 0.956（95%置信区间，95%CI，0.929∼0.982），其敏感性和特异性分别为 86.2%和 98.0%。3DMTM-SD 的 AUC 为 0.937（95%CI，0.909∼0.965），而 3DCM-SD 的 AUC 显著降低至 0.714（95%CI，0.649∼0.780），因为训练数据减少了。KNN-MMSD、LR-MMSD、SVM-MMSD 和 3DCM-MMSD 从纳入临床信息中显著受益，而使用较大数据集训练的模型在 COVID-19 鉴别方面仅略有性能提高。使用 CT 或多模态数据训练的 3DMTM 在 COVID-19 鉴别中表现出出色的稳健性。
3DMTM 算法在使用有限 CT 数据进行 COVID-19 鉴别方面表现出出色的稳健性。基于 CT 数据的 3DMTM 在 COVID-19 鉴别中的性能与基于多模态信息的性能相当。临床信息可以提高 KNN、LR、SVM 和 3DCM 在 COVID-19 鉴别中的性能，特别是在训练数据有限的情况下。
