Caruso Camillo Maria, Guarrasi Valerio, Cordelli Ermanno, Sicilia Rosa, Gentile Silvia, Messina Laura, Fiore Michele, Piccolo Claudia, Beomonte Zobel Bruno, Iannello Giulio, Ramella Sara, Soda Paolo
Research Unit of Computer Systems and Bioinformatics, Department of Engineering, Università Campus Bio-Medico di Roma, Via Àlvaro del Portillo, 21, 00128 Roma, Italy.
Department of Computer, Control, and Management Engineering, Sapienza University of Rome, 00185 Roma, Italy.
J Imaging. 2022 Nov 2;8(11):298. doi: 10.3390/jimaging8110298.
Lung cancer accounts for more deaths worldwide than any other cancer disease. In order to provide patients with the most effective treatment for these aggressive tumours, multimodal learning is emerging as a new and promising field of research that aims to extract complementary information from the data of different modalities for prognostic and predictive purposes. This knowledge could be used to optimise current treatments and maximise their effectiveness. To predict overall survival, in this work, we investigate the use of multimodal learning on the CLARO dataset, which includes CT images and clinical data collected from a cohort of non-small-cell lung cancer patients. Our method allows the identification of the optimal set of classifiers to be included in the ensemble in a late fusion approach. Specifically, after training unimodal models on each modality, it selects the best ensemble by solving a multiobjective optimisation problem that maximises both the recognition performance and the diversity of the predictions. In the ensemble, the labels of each sample are assigned using the majority voting rule. As further validation, we show that the proposed ensemble outperforms the models learning a single modality, obtaining state-of-the-art results on the task at hand.
在全球范围内，肺癌导致的死亡人数比其他任何癌症都要多。为了为这些侵袭性肿瘤患者提供最有效的治疗方法，多模态学习正在成为一个新的且有前景的研究领域，其旨在从不同模态的数据中提取互补信息，用于预后和预测目的。这些知识可用于优化当前治疗方法并使其效果最大化。为了预测总生存期，在这项工作中，我们研究了在CLARO数据集上使用多模态学习，该数据集包括从一组非小细胞肺癌患者中收集的CT图像和临床数据。我们的方法允许在后期融合方法中识别集成中要包含的最优分类器集。具体来说，在对每个模态训练单模态模型后，它通过解决一个多目标优化问题来选择最佳集成，该问题同时最大化识别性能和预测的多样性。在集成中，每个样本的标签使用多数投票规则进行分配。作为进一步的验证，我们表明所提出的集成优于学习单一模态的模型，在手边的任务上获得了当前最优的结果。
