Pati Pushpak, Karkampouna Sofia, Bonollo Francesco, Compérat Eva, Radić Martina, Spahn Martin, Martinelli Adriano, Wartenberg Martin, Kruithof-de Julio Marianna, Rapsomaniki Marianna
IBM Research Europe, Rüschlikon, Switzerland.
Urology Research Laboratory, Department for BioMedical Research, University of Bern, Bern, Switzerland.
Nat Mach Intell. 2024;6(9):1077-1093. doi: 10.1038/s42256-024-00889-5. Epub 2024 Sep 9.
Understanding the spatial heterogeneity of tumours and its links to disease initiation and progression is a cornerstone of cancer biology. Presently, histopathology workflows heavily rely on hematoxylin and eosin and serial immunohistochemistry staining, a cumbersome, tissue-exhaustive process that results in non-aligned tissue images. We propose the VirtualMultiplexer, a generative artificial intelligence toolkit that effectively synthesizes multiplexed immunohistochemistry images for several antibody markers (namely AR, NKX3.1, CD44, CD146, p53 and ERG) from only an input hematoxylin and eosin image. The VirtualMultiplexer captures biologically relevant staining patterns across tissue scales without requiring consecutive tissue sections, image registration or extensive expert annotations. Thorough qualitative and quantitative assessment indicates that the VirtualMultiplexer achieves rapid, robust and precise generation of virtually multiplexed imaging datasets of high staining quality that are indistinguishable from the real ones. The VirtualMultiplexer is successfully transferred across tissue scales and patient cohorts with no need for model fine-tuning. Crucially, the virtually multiplexed images enabled training a graph transformer that simultaneously learns from the joint spatial distribution of several proteins to predict clinically relevant endpoints. We observe that this multiplexed learning scheme was able to greatly improve clinical prediction, as corroborated across several downstream tasks, independent patient cohorts and cancer types. Our results showcase the clinical relevance of artificial intelligence-assisted multiplexed tumour imaging, accelerating histopathology workflows and cancer biology.
了解肿瘤的空间异质性及其与疾病发生和进展的联系是癌症生物学的基石。目前,组织病理学工作流程严重依赖苏木精和伊红染色以及连续免疫组织化学染色,这是一个繁琐、耗尽组织的过程,会导致组织图像不对齐。我们提出了VirtualMultiplexer,这是一种生成式人工智能工具包,它仅从输入的苏木精和伊红图像就能有效地合成针对几种抗体标记物(即AR、NKX3.1、CD44、CD146、p53和ERG)的多重免疫组织化学图像。VirtualMultiplexer能够在不连续的组织切片、图像配准或大量专家注释的情况下,捕捉跨组织尺度的生物学相关染色模式。全面的定性和定量评估表明,VirtualMultiplexer能够快速、稳健且精确地生成高质量染色的虚拟多重成像数据集,这些数据集与真实数据集难以区分。VirtualMultiplexer可以成功地跨组织尺度和患者队列进行转移,无需对模型进行微调。至关重要的是,虚拟多重图像能够训练一个图变换器,该变换器可以同时从几种蛋白质的联合空间分布中学习,以预测临床相关终点。我们观察到,这种多重学习方案能够极大地改善临床预测,这在多个下游任务、独立患者队列和癌症类型中都得到了证实。我们的结果展示了人工智能辅助多重肿瘤成像的临床相关性,加速了组织病理学工作流程和癌症生物学研究。