Zhang Yu-Dong, Zhang Zheng, Zhang Xin, Wang Shui-Hua
School of Informatics, University of Leicester, Leicester, LE1 7RH, UK.
Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen 518055, China.
Pattern Recognit Lett. 2021 Oct;150:8-16. doi: 10.1016/j.patrec.2021.06.021. Epub 2021 Jul 14.
COVID-19 has caused 3.34m deaths till 13/May/2021. It is now still causing confirmed cases and ongoing deaths every day.
This study investigated whether fusing chest CT with chest X-ray can help improve the AI's diagnosis performance. Data harmonization is employed to make a homogeneous dataset. We create an end-to-end multiple-input deep convolutional attention network (MIDCAN) by using the convolutional block attention module (CBAM). One input of our model receives 3D chest CT image, and other input receives 2D X-ray image. Besides, multiple-way data augmentation is used to generate fake data on training set. Grad-CAM is used to give explainable heatmap.
The proposed MIDCAN achieves a sensitivity of 98.10±1.88%, a specificity of 97.95±2.26%, and an accuracy of 98.02±1.35%.
Our MIDCAN method provides better results than 8 state-of-the-art approaches. We demonstrate the using multiple modalities can achieve better results than individual modality. Also, we demonstrate that CBAM can help improve the diagnosis performance.
截至2021年5月13日，新型冠状病毒肺炎（COVID-19）已导致334万人死亡。目前，它仍在每日引发确诊病例和持续死亡。
本研究调查了将胸部CT与胸部X线融合是否有助于提高人工智能的诊断性能。采用数据协调来创建一个同质化数据集。我们通过使用卷积块注意力模块（CBAM）创建了一个端到端多输入深度卷积注意力网络（MIDCAN）。我们模型的一个输入接收三维胸部CT图像，另一个输入接收二维X线图像。此外，采用多方式数据增强在训练集上生成虚假数据。使用梯度加权类激活映射（Grad-CAM）给出可解释的热图。
所提出的MIDCAN实现了98.10±1.88%的灵敏度、97.95±2.26%的特异性和98.02±1.35%的准确率。
我们的MIDCAN方法比8种先进方法提供了更好的结果。我们证明，使用多种模态比单一模态能取得更好的结果。此外，我们证明CBAM有助于提高诊断性能。
