Hu Yongfei, Li Xinyu, Yi Ying, Huang Yan, Wang Guangyu, Wang Dong
Dermatology Hospital, Southern Medical University, No. 2, Lujing Road, Yuexiu District, Guangzhou 510091, China.
Department of Bioinformatics, School of Basic Medical Sciences, Guangdong Province Key Laboratory of Molecular Tumor Pathology, Southern Medical University, 1023 Shatai South Road, Baiyun District, Guangzhou 510515, China.
Brief Bioinform. 2025 Mar 4;26(2). doi: 10.1093/bib/bbaf121.
Accurate cancer prognosis is essential for personalized clinical management, guiding treatment strategies and predicting patient survival. Conventional methods, which depend on the subjective evaluation of histopathological features, exhibit significant inter-observer variability and limited predictive power. To overcome these limitations, we developed cross-attention transformer-based multimodal fusion network (CATfusion), a deep learning framework that integrates multimodal histology-genomic data for comprehensive cancer survival prediction. By employing self-supervised learning strategy with TabAE for feature extraction and utilizing cross-attention mechanisms to fuse diverse data types, including mRNA-seq, miRNA-seq, copy number variation, DNA methylation variation, mutation data, and histopathological images. By successfully integrating this multi-tiered patient information, CATfusion has become an advanced survival prediction model to utilize the most diverse data types across various cancer types. CATfusion's architecture, which includes a bidirectional multimodal attention mechanism and self-attention block, is adept at synchronizing the learning and integration of representations from various modalities. CATfusion achieves superior predictive performance over traditional and unimodal models, as demonstrated by enhanced C-index and survival area under the curve scores. The model's high accuracy in stratifying patients into distinct risk groups is a boon for personalized medicine, enabling tailored treatment plans. Moreover, CATfusion's interpretability, enabled by attention-based visualization, offers insights into the biological underpinnings of cancer prognosis, underscoring its potential as a transformative tool in oncology.
准确的癌症预后对于个性化临床管理、指导治疗策略和预测患者生存至关重要。传统方法依赖于对组织病理学特征的主观评估，存在显著的观察者间差异和有限的预测能力。为了克服这些局限性，我们开发了基于交叉注意力变换器的多模态融合网络（CATfusion），这是一个深度学习框架，它整合多模态组织学-基因组数据以进行全面的癌症生存预测。通过采用带有TabAE的自监督学习策略进行特征提取，并利用交叉注意力机制融合多种数据类型，包括mRNA序列、miRNA序列、拷贝数变异、DNA甲基化变异、突变数据和组织病理学图像。通过成功整合这种多层次的患者信息，CATfusion已成为一种先进的生存预测模型，可利用各种癌症类型中最多样化的数据类型。CATfusion的架构包括双向多模态注意力机制和自注意力模块，擅长同步来自各种模态的表征的学习和整合。如增强的C指数和曲线下生存面积得分所示，CATfusion在预测性能上优于传统模型和单模态模型。该模型在将患者分层为不同风险组方面的高准确性对个性化医疗有益，能够制定量身定制的治疗方案。此外，通过基于注意力的可视化实现的CATfusion的可解释性，为癌症预后的生物学基础提供了见解，突出了其作为肿瘤学中变革性工具的潜力。
