Advanced Institute of Information Technology, Peking University, Hangzhou, Zhejiang 310000, China.
National Engineering Research Center of Visual Technology, National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing 100871, China.
Bioinformatics. 2023 Aug 1;39(8). doi: 10.1093/bioinformatics/btad464.
Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners.
Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low- and high-resolution EM image datasets as well as other natural image datasets.
The code and dataset can be found at https://github.com/EmmaSRH/PS-Net.
在电子显微镜 (EM) 图像中进行细胞膜分割是 EM 图像处理的关键步骤。然而，虽然流行的方法在低分辨率 EM 数据集上的性能可与人类相媲美，但在应用于高分辨率 EM 数据集时，它们的表现却受到限制。相比之下，人类视觉系统在低分辨率和高分辨率下都表现出一致的出色性能。为了更好地理解这一限制，我们进行了眼动和感知一致性实验。我们的数据表明，与常用的评估标准相反，人类观察者在容忍膜不对齐的情况下，对膜的结构更为敏感。此外，我们的结果表明，人类视觉系统以全局-局部和粗到细的方式处理图像。
基于这些观察，我们提出了一种包含人类感知特征的细胞膜分割计算框架。该框架包括一种新的评估指标——感知 Hausdorff 距离 (PHD)，以及一个名为 PHD 引导分割网络 (PS-Net) 的端到端网络，该网络使用自适应调整的 PHD 损失函数和多尺度架构进行训练。我们的主观实验表明，与其他标准相比，PHD 指标与人类感知更为一致，我们提出的 PS-Net 在低分辨率和高分辨率 EM 图像数据集以及其他自然图像数据集上的表现均优于最先进的方法。
代码和数据集可在 https://github.com/EmmaSRH/PS-Net 上找到。
