Zhou Yanfeng, Li Lingrui, Wang Chenlong, Song Le, Yang Ge
IEEE Trans Med Imaging. 2025 Feb;44(2):1058-1069. doi: 10.1109/TMI.2024.3474028. Epub 2025 Feb 4.
Semantic segmentation of electron microscopy (EM) images is crucial for nanoscale analysis. With the development of deep neural networks (DNNs), semantic segmentation of EM images has achieved remarkable success. However, current EM image segmentation models are usually extensions or adaptations of natural or biomedical models. They lack the full exploration and utilization of the intrinsic characteristics of EM images. Furthermore, they are often designed only for several specific segmentation objects and lack versatility. In this study, we quantitatively analyze the characteristics of EM images compared with those of natural and other biomedical images via the wavelet transform. To better utilize these characteristics, we design a high-frequency (HF) fusion network, GobletNet, which outperforms state-of-the-art models by a large margin in the semantic segmentation of EM images. We use the wavelet transform to generate HF images as extra inputs and use an extra encoding branch to extract HF information. Furthermore, we introduce a fusion-attention module (FAM) into GobletNet to facilitate better absorption and fusion of information from raw images and HF images. Extensive benchmarking on seven public EM datasets (EPFL, CREMI, SNEMI3D, UroCell, MitoEM, Nanowire and BetaSeg) demonstrates the effectiveness of our model. The code is available at https://github.com/Yanfeng-Zhou/GobletNet.
电子显微镜（EM）图像的语义分割对于纳米尺度分析至关重要。随着深度神经网络（DNN）的发展，EM图像的语义分割取得了显著成功。然而，当前的EM图像分割模型通常是自然或生物医学模型的扩展或改编。它们缺乏对EM图像内在特征的充分探索和利用。此外，它们通常仅针对几个特定的分割对象进行设计，缺乏通用性。在本研究中，我们通过小波变换定量分析了EM图像与自然图像和其他生物医学图像相比的特征。为了更好地利用这些特征，我们设计了一种高频（HF）融合网络GobletNet，它在EM图像的语义分割方面比现有最先进的模型有大幅提升。我们使用小波变换生成高频图像作为额外输入，并使用一个额外的编码分支来提取高频信息。此外，我们在GobletNet中引入了一个融合注意力模块（FAM），以更好地吸收和融合来自原始图像和高频图像的信息。在七个公共EM数据集（EPFL、CREMI、SNEMI3D、UroCell、MitoEM、Nanowire和BetaSeg）上进行的广泛基准测试证明了我们模型的有效性。代码可在https://github.com/Yanfeng-Zhou/GobletNet获取。
