IEEE Trans Med Imaging. 2023 Aug;42(8):2462-2473. doi: 10.1109/TMI.2023.3253760. Epub 2023 Aug 1.
Cancer survival prediction requires exploiting related multimodal information (e.g., pathological, clinical and genomic features, etc.) and it is even more challenging in clinical practices due to the incompleteness of patient's multimodal data. Furthermore, existing methods lack sufficient intra- and inter-modal interactions, and suffer from significant performance degradation caused by missing modalities. This manuscript proposes a novel hybrid graph convolutional network, entitled HGCN, which is equipped with an online masked autoencoder paradigm for robust multimodal cancer survival prediction. Particularly, we pioneer modeling the patient's multimodal data into flexible and interpretable multimodal graphs with modality-specific preprocessing. HGCN integrates the advantages of graph convolutional networks (GCNs) and a hypergraph convolutional network (HCN) through node message passing and a hyperedge mixing mechanism to facilitate intra-modal and inter-modal interactions between multimodal graphs. With HGCN, the potential for multimodal data to create more reliable predictions of patient's survival risk is dramatically increased compared to prior methods. Most importantly, to compensate for missing patient modalities in clinical scenarios, we incorporated an online masked autoencoder paradigm into HGCN, which can effectively capture intrinsic dependence between modalities and seamlessly generate missing hyperedges for model inference. Extensive experiments and analysis on six cancer cohorts from TCGA show that our method significantly outperforms the state-of-the-arts in both complete and missing modal settings. Our codes are made available at https://github.com/lin-lcx/HGCN.
癌症生存预测需要利用相关的多模态信息(例如,病理、临床和基因组特征等),由于患者多模态数据的不完整,在临床实践中这更加具有挑战性。此外,现有的方法缺乏充分的内在和模态间的相互作用,并且由于模态的缺失而导致性能显著下降。本文提出了一种新颖的混合图卷积网络,称为 HGCN,它配备了在线掩蔽自动编码器范例,用于稳健的多模态癌症生存预测。特别是,我们开创性地将患者的多模态数据建模为具有特定模态预处理的灵活和可解释的多模态图。HGCN 通过节点消息传递和超边混合机制集成了图卷积网络(GCN)和超图卷积网络(HCN)的优点,以促进多模态图之间的内在模态和模态间的相互作用。与之前的方法相比,HGCN 极大地提高了多模态数据对患者生存风险进行更可靠预测的潜力。最重要的是,为了在临床场景中弥补患者模态的缺失,我们将在线掩蔽自动编码器范例纳入 HGCN 中,该范例可以有效地捕获模态之间的内在依赖关系,并为模型推断无缝生成缺失的超边。在来自 TCGA 的六个癌症队列上进行的广泛实验和分析表明,我们的方法在完整和缺失模态设置下都显著优于最先进的方法。我们的代码可在 https://github.com/lin-lcx/HGCN 上获得。