Laboratorio Biostatistica e Bioinformatica, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Bari, Italy.
Servizio di Fisica Sanitaria, Azienda Ospedaliero-Universitaria di Parma, Parma, Italy.
PLoS One. 2024 Sep 10;19(9):e0303217. doi: 10.1371/journal.pone.0303217. eCollection 2024.
To address the numerous unmeet clinical needs, in recent years several Machine Learning models applied to medical images and clinical data have been introduced and developed. Even when they achieve encouraging results, they lack evolutionary progression, thus perpetuating their status as autonomous entities. We postulated that different algorithms which have been proposed in the literature to address the same diagnostic task, can be aggregated to enhance classification performance. We suggested a proof of concept to define an ensemble approach useful for integrating different algorithms proposed to solve the same clinical task.
The proposed approach was developed starting from a public database consisting of radiomic features extracted from CT images relating to 535 patients suffering from lung cancer. Seven algorithms were trained independently by participants in the AI4MP working group on Artificial Intelligence of the Italian Association of Physics in Medicine to discriminate metastatic from non-metastatic patients. The classification scores generated by these algorithms are used to train SVM classifier. The Explainable Artificial Intelligence approach is applied to the final model. The ensemble model was validated following an 80-20 hold-out and leave-one-out scheme on the training set.
Compared to individual algorithms, a more accurate result was achieved. On the independent test the ensemble model achieved an accuracy of 0.78, a F1-score of 0.57 and a log-loss of 0.49. Shapley values representing the contribution of each algorithm to the final classification result of the ensemble model were calculated. This information represents an added value for the end user useful for evaluating the appropriateness of the classification result on a particular case. It also allows us to evaluate on a global level which methodological approaches of the individual algorithms are likely to have the most impact.
Our proposal represents an innovative approach useful for integrating different algorithms that populate the literature and which lays the foundations for future evaluations in broader application scenarios.
为了满足众多未满足的临床需求，近年来，已经引入和开发了一些应用于医学图像和临床数据的机器学习模型。即使它们取得了令人鼓舞的结果，它们也缺乏进化的进展，因此仍然是自主的实体。我们假设，已经在文献中提出的用于解决相同诊断任务的不同算法，可以被聚合以增强分类性能。我们提出了一个概念验证来定义一种有用的集成方法，用于集成为解决相同临床任务而提出的不同算法。
该方法是从一个公共数据库开始开发的，该数据库由与 535 名肺癌患者相关的 CT 图像提取的放射组学特征组成。七个算法由 AI4MP 工作组的参与者独立训练，以区分转移性和非转移性患者。这些算法生成的分类分数用于训练 SVM 分类器。应用可解释人工智能方法对最终模型进行解释。在训练集上，采用 80-20 分割和留一法对集成模型进行验证。
与单个算法相比，该方法取得了更准确的结果。在独立测试中，集成模型的准确率为 0.78，F1 得分为 0.57，对数损失为 0.49。计算了每个算法对集成模型最终分类结果的贡献的 Shapley 值。这些信息为最终用户提供了一个附加价值，有助于评估特定病例的分类结果的适当性。它还允许我们从全局水平评估个体算法的方法学方法最有可能产生影响的程度。
我们的建议代表了一种有用的创新方法，用于集成文献中存在的不同算法，并为未来在更广泛的应用场景中的评估奠定了基础。
