Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK.
Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, UK.
Med Image Anal. 2024 Jan;91:102995. doi: 10.1016/j.media.2023.102995. Epub 2023 Oct 11.
Automated synthesis of histology images has several potential applications in computational pathology. However, no existing method can generate realistic tissue images with a bespoke cellular layout or user-defined histology parameters. In this work, we propose a novel framework called SynCLay (Synthesis from Cellular Layouts) that can construct realistic and high-quality histology images from user-defined cellular layouts along with annotated cellular boundaries. Tissue image generation based on bespoke cellular layouts through the proposed framework allows users to generate different histological patterns from arbitrary topological arrangement of different types of cells (e.g., neutrophils, lymphocytes, epithelial cells and others). SynCLay generated synthetic images can be helpful in studying the role of different types of cells present in the tumor microenvironment. Additionally, they can assist in balancing the distribution of cellular counts in tissue images for designing accurate cellular composition predictors by minimizing the effects of data imbalance. We train SynCLay in an adversarial manner and integrate a nuclear segmentation and classification model in its training to refine nuclear structures and generate nuclear masks in conjunction with synthetic images. During inference, we combine the model with another parametric model for generating colon images and associated cellular counts as annotations given the grade of differentiation and cellularities (cell densities) of different cells. We assess the generated images quantitatively using the Frechet Inception Distance and report on feedback from trained pathologists who assigned realism scores to a set of images generated by the framework. The average realism score across all pathologists for synthetic images was as high as that for the real images. Moreover, with the assistance from pathologists, we showcase the ability of the generated images to accurately differentiate between benign and malignant tumors, thus reinforcing their reliability. We demonstrate that the proposed framework can be used to add new cells to a tissue images and alter cellular positions. We also show that augmenting limited real data with the synthetic data generated by our framework can significantly boost prediction performance of the cellular composition prediction task. The implementation of the proposed SynCLay framework is available at https://github.com/Srijay/SynCLay-Framework.
组织学图像的自动化合成在计算病理学中有几个潜在的应用。然而，目前没有任何方法可以生成具有特定细胞布局或用户定义的组织学参数的逼真组织图像。在这项工作中，我们提出了一个名为 SynCLay（基于细胞布局的合成）的新框架，该框架可以根据用户定义的细胞布局和带注释的细胞边界来构建逼真的高质量组织学图像。通过该框架基于定制细胞布局进行组织图像生成，允许用户从不同类型细胞的任意拓扑排列中生成不同的组织学模式（例如，中性粒细胞、淋巴细胞、上皮细胞等）。SynCLay 生成的合成图像有助于研究肿瘤微环境中不同类型细胞的作用。此外，它们可以通过最小化数据不平衡的影响来辅助平衡组织图像中细胞计数的分布，从而设计准确的细胞组成预测器。我们以对抗的方式训练 SynCLay，并在其训练中集成核分割和分类模型，以细化核结构并生成与合成图像一起的核掩模。在推理过程中，我们将该模型与另一个参数模型结合，用于生成结肠图像和相关的细胞计数作为不同细胞的分化程度和细胞密度（细胞密度）的注释。我们使用 Frechet Inception 距离对生成的图像进行定量评估，并报告经过训练的病理学家的反馈，他们为框架生成的一组图像分配了逼真度分数。所有病理学家对合成图像的平均逼真度评分都与真实图像一样高。此外，在病理学家的协助下，我们展示了生成的图像准确地区分良性和恶性肿瘤的能力，从而增强了其可靠性。我们证明了所提出的框架可用于向组织图像添加新细胞并改变细胞位置。我们还表明，用我们的框架生成的合成数据增强有限的真实数据可以显著提高细胞组成预测任务的预测性能。所提出的 SynCLay 框架的实现可在 https://github.com/Srijay/SynCLay-Framework 上获得。
