Murali Leema Krishna, Lutnick Brendon, Ginley Brandon, Tomaszewski John E, Sarder Pinaki
Department of Biomedical Engineering, SUNY Buffalo, Buffalo, NY, USA 14228.
Department of Pathology and Anatomical Sciences, SUNY Buffalo, Buffalo, NY, USA 14203.
Proc SPIE Int Soc Opt Eng. 2020 Feb;11320. doi: 10.1117/12.2549891. Epub 2020 Mar 16.
Generative adversarial networks (GANs) have received immense attention in the field of machine learning for their potential to learn high-dimensional and real data distribution. These methods do not rely on any assumptions about the data distribution of the input sample and can generate real-like samples from latent vector space based on unsupervised learning. In the medical field, particularly, in digital pathology expert annotation and availability of a large set of training data is costly and the study of manifestations of various diseases is based on visual examination of stained slides. In clinical practice, various staining information is required to improve the pathological diagnosis process. But when the sampled tissue to be examined is limited, the final diagnosis made by the pathologist is based on limited stain styles. These limitations can be overcome by studying the usability and reliability of generative models in the field of digital pathology. To understand the usability of the generative models, we synthesize in an unsupervised manner, high resolution renal microanatomical structures like renal glomerulus in thin tissue histology images using state-of-art architectures like Deep Convolutional Generative Adversarial Network (DCGAN) and Enhanced Super-Resolution Generative Adversarial Network (ESRGAN). Successful generation of such structures will lead to obtaining a large set of labeled data for further developing supervised algorithms for disease classification and understanding progression. Our study suggests while GAN is able to attain formalin fixed and paraffin embedded tissue image quality, GAN requires further prior knowledge as input to model intrinsic micro-anatomical details, such as capillary wall, urinary pole, nuclei placement, suggesting developing semi-supervised architectures by using these above details as prior information. Also, the generative models can be used to create an artificial effect of staining without physically tampering the histopathological slide. To demonstrate this, we use a CycleGAN network to transform Hematoxylin and eosin (H&E) stain to Periodic acid-Schiff (PAS) stain and Jones methenamine silver (JMS) stain to PAS stain. In this way GAN can be employed to translate different renal pathology stain styles when the relevant staining information is not available in the clinical settings.
生成对抗网络(GAN)因其学习高维真实数据分布的潜力而在机器学习领域受到了极大关注。这些方法不依赖于对输入样本数据分布的任何假设,并且可以基于无监督学习从潜在向量空间生成逼真的样本。特别是在医学领域,数字病理学中的专家注释和大量训练数据的获取成本高昂,并且各种疾病表现的研究基于对染色玻片的视觉检查。在临床实践中,需要各种染色信息来改进病理诊断过程。但是当待检查的采样组织有限时,病理学家做出的最终诊断基于有限的染色方式。通过研究生成模型在数字病理学领域的可用性和可靠性,可以克服这些限制。为了理解生成模型的可用性,我们使用深度卷积生成对抗网络(DCGAN)和增强超分辨率生成对抗网络(ESRGAN)等先进架构,以无监督方式在薄组织组织学图像中合成高分辨率的肾脏微观解剖结构,如肾小球。成功生成此类结构将导致获得大量标记数据,以进一步开发用于疾病分类和理解病情进展的监督算法。我们的研究表明,虽然GAN能够达到福尔马林固定石蜡包埋组织图像的质量,但GAN需要进一步的先验知识作为输入来建模内在的微观解剖细节,如毛细血管壁、尿极、细胞核位置,这表明可以通过使用上述细节作为先验信息来开发半监督架构。此外,生成模型可用于在不实际篡改组织病理玻片的情况下创建人工染色效果。为了证明这一点,我们使用循环GAN网络将苏木精和伊红(H&E)染色转换为过碘酸希夫(PAS)染色,并将琼斯甲胺银(JMS)染色转换为PAS染色。通过这种方式,当临床环境中没有相关染色信息时,GAN可用于转换不同的肾脏病理染色方式。