IEEE J Biomed Health Inform. 2021 Feb;25(2):403-411. doi: 10.1109/JBHI.2020.2975151. Epub 2021 Feb 5.
Stain virtualization is an application with growing interest in digital pathology allowing simulation of stained tissue images thus saving lab and tissue resources. Thanks to the success of Generative Adversarial Networks (GANs) and the progress of unsupervised learning, unsupervised style transfer GANs have been successfully used to generate realistic, clinically meaningful and interpretable images. The large size of high resolution Whole Slide Images (WSIs) presents an additional computational challenge. This makes tilewise processing necessary during training and inference of deep learning networks. Instance normalization has a substantial positive effect in style transfer GAN applications but with tilewise inference, it has the tendency to cause a tiling artifact in reconstructed WSIs. In this paper we propose a novel perceptual embedding consistency (PEC) loss forcing the network to learn color, contrast and brightness invariant features in the latent space and hence substantially reducing the aforementioned tiling artifact. Our approach results in more seamless reconstruction of the virtual WSIs. We validate our method quantitatively by comparing the virtually generated images to their corresponding consecutive real stained images. We compare our results to state-of-the-art unsupervised style transfer methods and to the measures obtained from consecutive real stained tissue slide images. We demonstrate our hypothesis about the effect of the PEC loss by comparing model robustness to color, contrast and brightness perturbations and visualizing bottleneck embeddings. We validate the robustness of the bottleneck feature maps by measuring their sensitivity to the different perturbations and using them in a tumor segmentation task. Additionally, we propose a preliminary validation of the virtual staining application by comparing interpretation of 2 pathologists on real and virtual tiles and inter-pathologist agreement.
染色虚拟化为数字病理学领域日益受到关注的应用程序，它允许模拟染色组织图像，从而节省实验室和组织资源。得益于生成对抗网络（GAN）的成功和无监督学习的进展，无监督风格迁移 GAN 已成功用于生成真实、有临床意义且可解释的图像。高分辨率全切片图像（WSI）的大型尺寸带来了额外的计算挑战。这使得在深度学习网络的训练和推理过程中需要进行瓦片处理。实例归一化在风格迁移 GAN 应用中具有显著的积极效果，但在瓦片推理中，它有在重构 WSI 中产生瓦片伪影的趋势。在本文中，我们提出了一种新颖的感知嵌入一致性（PEC）损失，迫使网络在潜在空间中学习颜色、对比度和亮度不变的特征，从而大大减少上述瓦片伪影。我们的方法可以更无缝地重建虚拟 WSI。我们通过将虚拟生成的图像与其对应的连续真实染色图像进行比较，从定量上验证了我们的方法。我们将我们的结果与最先进的无监督风格迁移方法以及从连续的真实染色组织幻灯片图像中获得的测量值进行了比较。我们通过比较模型对颜色、对比度和亮度扰动的鲁棒性以及可视化瓶颈嵌入来验证 PEC 损失的效果假设。我们通过测量不同扰动对瓶颈特征图的敏感性并将其用于肿瘤分割任务来验证瓶颈特征图的鲁棒性。此外，我们通过比较两位病理学家对真实和虚拟瓦片的解释以及病理学家之间的一致性，对虚拟染色应用进行了初步验证。
