Zhang Zhehao, Gao Linlin, Jin Guang, Guo Lijun, Yao Yudong, Dong Li, Han Jinming
Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China.
Research Institute for Medical and Biological Engineering, Ningbo University, Ningbo, China.
Quant Imaging Med Surg. 2021 Jul;11(7):3338-3354. doi: 10.21037/qims-21-91.
To assist doctors to diagnose mild cognitive impairment (MCI) and Alzheimer's disease (AD) early and accurately, convolutional neural networks based on structural magnetic resonance imaging (sMRI) images have been developed and shown excellent performance. However, they are still limited in their capacity in extracting discriminative features because of large sMRI image volumes yet small lesion regions and the small number of sMRI images.
We proposed a task-driven hierarchical attention network (THAN) taking advantage of the merits of patch-based and attention-based convolutional neural networks for MCI and AD diagnosis. THAN consists of an information sub-network and a hierarchical attention sub-network. In the information sub-network, an information map extractor, a patch-assistant module, and a mutual-boosting loss function are designed to generate a task-driven information map, which automatically highlights disease-related regions and their importance for final classification. In the hierarchical attention sub-network, a visual attention module and a semantic attention module are devised based on the information map to extract discriminative features for disease diagnosis.
Extensive experiments were conducted for four classification tasks: MCI versus () normal controls (NC), AD NC, AD MCI, and AD MCI NC. Results demonstrated that THAN attained the accuracy of 81.6% for MCI NC, 93.5% for AD NC, 80.8% for AD MCI, and 62.9% for AD MCI NC. It outperformed advanced attention-based and patch-based methods. Moreover, information maps generated by the information sub-network could highlight the potential biomarkers of MCI and AD, such as the hippocampus and ventricles. Furthermore, when the visual and semantic attention modules were combined, the performance of the four tasks was highly improved.
The information sub-network can automatically highlight the disease-related regions. The hierarchical attention sub-network can extract discriminative visual and semantic features. Through the two sub-networks, THAN fully exploits the visual and semantic features of disease-related regions and meanwhile considers global features of sMRI images, which finally facilitate the diagnosis of MCI and AD.
为帮助医生早期准确诊断轻度认知障碍（MCI）和阿尔茨海默病（AD），基于结构磁共振成像（sMRI）图像的卷积神经网络已被开发出来并表现出优异性能。然而，由于sMRI图像体积大但病变区域小且sMRI图像数量少，它们在提取判别特征的能力上仍存在局限。
我们提出了一种任务驱动的分层注意力网络（THAN），利用基于补丁和基于注意力的卷积神经网络的优点来进行MCI和AD诊断。THAN由一个信息子网络和一个分层注意力子网络组成。在信息子网络中，设计了一个信息图提取器、一个补丁辅助模块和一个相互增强损失函数，以生成一个任务驱动的信息图，该图能自动突出与疾病相关的区域及其对最终分类的重要性。在分层注意力子网络中，基于信息图设计了一个视觉注意力模块和一个语义注意力模块，以提取用于疾病诊断的判别特征。
针对四个分类任务进行了广泛实验：MCI与正常对照（NC）、AD与NC、AD与MCI以及AD与MCI与NC。结果表明，THAN在MCI与NC任务中准确率达到81.6%，AD与NC任务中为93.5%，AD与MCI任务中为80.8%，AD与MCI与NC任务中为62.9%。它优于先进的基于注意力和基于补丁的方法。此外，信息子网络生成的信息图可以突出MCI和AD的潜在生物标志物，如海马体和脑室。此外，当视觉和语义注意力模块结合时，四个任务的性能得到了显著提高。
信息子网络可以自动突出与疾病相关的区域。分层注意力子网络可以提取有判别力的视觉和语义特征。通过这两个子网络，THAN充分利用了与疾病相关区域的视觉和语义特征，同时考虑了sMRI图像的全局特征，最终有助于MCI和AD的诊断。
