Institute of Biomedical Engineering, School of Communication and Information Engineering, Shanghai University, Shanghai, China.
Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.
Med Phys. 2022 Jan;49(1):144-157. doi: 10.1002/mp.15341. Epub 2021 Nov 29.
Recent studies have illustrated that the peritumoral regions of medical images have value for clinical diagnosis. However, the existing approaches using peritumoral regions mainly focus on the diagnostic capability of the single region and ignore the advantages of effectively fusing the intratumoral and peritumoral regions. In addition, these methods need accurate segmentation masks in the testing stage, which are tedious and inconvenient in clinical applications. To address these issues, we construct a deep convolutional neural network that can adaptively fuse the information of multiple tumoral-regions (FMRNet) for breast tumor classification using ultrasound (US) images without segmentation masks in the testing stage.
To sufficiently excavate the potential relationship, we design a fused network and two independent modules to extract and fuse features of multiple regions simultaneously. First, we introduce two enhanced combined-tumoral (EC) region modules, aiming to enhance the combined-tumoral features gradually. Then, we further design a three-branch module for extracting and fusing the features of intratumoral, peritumoral, and combined-tumoral regions, denoted as the intratumoral, peritumoral, and combined-tumoral module. Especially, we design a novel fusion module by introducing a channel attention module to adaptively fuse the features of three regions. The model is evaluated on two public datasets including UDIAT and BUSI with breast tumor ultrasound images. Two independent groups of experiments are performed on two respective datasets using the fivefold stratified cross-validation strategy. Finally, we conduct ablation experiments on two datasets, in which BUSI is used as the training set and UDIAT is used as the testing set.
We conduct detailed ablation experiments about the proposed two modules and comparative experiments with other existing representative methods. The experimental results show that the proposed method yields state-of-the-art performance on both two datasets. Especially, in the UDIAT dataset, the proposed FMRNet achieves a high accuracy of 0.945 and a specificity of 0.945, respectively. Moreover, the precision (PRE = 0.909) even dramatically improves by 21.6% on the BUSI dataset compared with the existing method of the best result.
The proposed FMRNet shows good performance in breast tumor classification with US images, and proves its capability of exploiting and fusing the information of multiple tumoral-regions. Furthermore, the FMRNet has potential value in classifying other types of cancers using multiple tumoral-regions of other kinds of medical images.
最近的研究表明，医学图像的瘤周区域对临床诊断具有价值。然而，现有的利用瘤周区域的方法主要侧重于单个区域的诊断能力，而忽略了有效融合肿瘤内和瘤周区域的优势。此外，这些方法在测试阶段需要准确的分割掩模，这在临床应用中既繁琐又不方便。为了解决这些问题，我们构建了一个深度卷积神经网络，可以在没有分割掩模的情况下，自适应地融合多个肿瘤区域的信息（FMRNet），用于使用超声（US）图像进行乳腺癌分类。
为了充分挖掘潜在的关系，我们设计了一个融合网络和两个独立的模块，以同时提取和融合多个区域的特征。首先，我们引入了两个增强型联合肿瘤（EC）区域模块，旨在逐步增强联合肿瘤的特征。然后，我们进一步设计了一个三分支模块，用于提取和融合肿瘤内、瘤周和联合肿瘤区域的特征，分别表示为肿瘤内、瘤周和联合肿瘤模块。特别是，我们通过引入通道注意力模块设计了一个新颖的融合模块，以自适应地融合三个区域的特征。该模型在包含乳腺癌超声图像的两个公共数据集 UDIAT 和 BUSI 上进行评估。使用五重分层交叉验证策略在两个各自的数据集上进行了两组独立的实验。最后，我们在两个数据集上进行了消融实验，其中 BUSI 用作训练集，UDIAT 用作测试集。
我们对所提出的两个模块进行了详细的消融实验，并与其他现有代表性方法进行了对比实验。实验结果表明，该方法在两个数据集上均取得了最先进的性能。特别是在 UDIAT 数据集上，所提出的 FMRNet 分别达到了 0.945 的高准确率和 0.945 的高特异性。此外，与现有最佳结果方法相比，BUSI 数据集上的精度（PRE=0.909）甚至大幅提高了 21.6%。
所提出的 FMRNet 在使用 US 图像进行乳腺癌分类方面表现出良好的性能，证明了其利用和融合多个肿瘤区域信息的能力。此外，FMRNet 在使用其他类型医学图像的多个肿瘤区域对其他类型癌症进行分类方面具有潜在价值。
