School of Information Engineering, East China University of Technology, Nanchang, China.
School of Software, East China University of Technology, Nanchang, China.
Comput Methods Biomech Biomed Engin. 2024 May;27(7):883-896. doi: 10.1080/10255842.2023.2211188. Epub 2023 May 11.
Accurate survival prediction is a critical goal in the prognosis of breast cancer patients because it can help physicians make more patient-friendly decisions and further guide appropriate treatment. Breast cancer is often caused by genetic abnormalities, which prompts researchers to consider information such as gene expression and copy number variation in addition to clinical data in their studies. The integration of these multi-modal data can improve the predictive power of models. However, with the highly unbalanced information of breast cancer patient data, it becomes a new challenge for breast cancer patient survival prediction to fully extract the characteristic information of these multi-modal data and to consider the complementarity of this information. To this end, we propose a deep multi-modal fusion network (DMMFN) to predict the five-year survival of breast cancer patients by integrating clinical data, copy number variation data, and gene expression data. The imbalanced dataset is first processed using the oversampling method SMOTE-NC. Then the abstract modal features of the multi-modal data are extracted by the two-layer one-dimensional convolutional neural network and the bi-directional long short-term memory network. Next, the weight coefficients of each modal data are dynamically adjusted using gated multimodal units to obtain fusion features. Finally, the fusion features are fed into the MaxoutMLP classifier to obtain the final prediction results. We conducted experiments on the METABRIC dataset to verify the validity of the multi-modal data and compared it with other methods. The comprehensive performance evaluation shows that DMMFN has better prediction performance.
准确的生存预测是乳腺癌患者预后的一个关键目标，因为它可以帮助医生做出更有利于患者的决策，并进一步指导适当的治疗。乳腺癌通常是由基因异常引起的，这促使研究人员在研究中除了考虑临床数据外，还考虑基因表达和拷贝数变异等信息。这些多模态数据的整合可以提高模型的预测能力。然而，由于乳腺癌患者数据的信息高度不平衡，充分提取这些多模态数据的特征信息并考虑这些信息的互补性，成为乳腺癌患者生存预测的一个新挑战。为此，我们提出了一种深度多模态融合网络（DMMFN），通过整合临床数据、拷贝数变异数据和基因表达数据来预测乳腺癌患者的五年生存率。首先使用过采样方法 SMOTE-NC 处理不平衡数据集。然后，通过两层一维卷积神经网络和双向长短期记忆网络提取多模态数据的抽象模态特征。接下来，使用门控多模态单元动态调整每个模态数据的权重系数，以获得融合特征。最后，将融合特征输入到 MaxoutMLP 分类器中，得到最终的预测结果。我们在 METABRIC 数据集上进行了实验，验证了多模态数据的有效性，并与其他方法进行了比较。综合性能评估表明，DMMFN 具有更好的预测性能。
