Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, 94305, USA.
Department of Radiation Oncology, Wake Forest University School of Medicine, Winston Salem, NC, 27109, USA.
EBioMedicine. 2024 Sep;107:105287. doi: 10.1016/j.ebiom.2024.105287. Epub 2024 Aug 17.
Multiplexed immunofluorescence (mIF) staining, such as CODEX and MIBI, holds significant clinical value for various fields, such as disease diagnosis, biological research, and drug development. However, these techniques are often hindered by high time and cost requirements.
Here we present a Multimodal-Attention-based virtual mIF Staining (MAS) system that utilises a deep learning model to extract potential antibody-related features from dual-modal non-antibody-stained fluorescence imaging, specifically autofluorescence (AF) and DAPI imaging. The MAS system simultaneously generates predictions of mIF with multiple survival-associated biomarkers in gastric cancer using self- and multi-attention learning mechanisms.
Experimental results with 180 pathological slides from 94 patients with gastric cancer demonstrate the efficiency and consistent performance of the MAS system in both cancer and noncancer gastric tissues. Furthermore, we showcase the prognostic accuracy of the virtual mIF images of seven gastric cancer related biomarkers, including CD3, CD20, FOXP3, PD1, CD8, CD163, and PD-L1, which is comparable to those obtained from the standard mIF staining.
The MAS system rapidly generates reliable multiplexed staining, greatly reducing the cost of mIF and improving clinical workflow.
Stanford 2022 HAI Seed Grant; National Institutes of Health 1R01CA256890.
多重免疫荧光（mIF）染色，如 CODEX 和 MIBI，在疾病诊断、生物研究和药物开发等多个领域具有重要的临床价值。然而，这些技术通常受到时间和成本要求高的限制。
在这里，我们提出了一种基于多模态注意的虚拟 mIF 染色（MAS）系统，该系统利用深度学习模型从双模态非抗体染色荧光成像（即自发荧光（AF）和 DAPI 成像）中提取潜在的抗体相关特征。MAS 系统利用自注意和多注意学习机制，同时对来自 94 名胃癌患者的 180 张病理切片进行预测，生成与多个与生存相关的生物标志物相关的 mIF 图像。
来自 94 名胃癌患者的 180 张病理切片的实验结果表明，MAS 系统在癌症和非癌症胃组织中均具有高效和一致的性能。此外，我们展示了七个与胃癌相关的生物标志物（包括 CD3、CD20、FOXP3、PD1、CD8、CD163 和 PD-L1）的虚拟 mIF 图像的预后准确性，与标准 mIF 染色获得的结果相当。
MAS 系统快速生成可靠的多重染色，大大降低了 mIF 的成本，提高了临床工作流程的效率。
斯坦福 2022 HAI 种子基金；美国国立卫生研究院 1R01CA256890。
