Alinia Shayeste, Asghari-Jafarabadi Mohammad, Mahmoudi Leila, Roshanaei Ghodratollah, Safari Maliheh
Department of Statistics and Epidemiology, School of Medicine, Zanjan University of Medical Sciences, Zanjan, Iran.
Road Traffic Injury Research Center, Tabriz University of Medical Sciences, Tabriz, Iran.
Heliyon. 2024 Mar 12;10(6):e27854. doi: 10.1016/j.heliyon.2024.e27854. eCollection 2024 Mar 30.
Colorectal cancer (CRC), also known as colorectal cancer, is a significant disease marked by high fatality rates, ranking as the third leading cause of global mortality. The main objective of this study was to assess the accuracy of predictive models in predicting both mortality events and the probability of disease recurrence.
A retrospective analysis was conducted on a cohort of 284 individuals diagnosed with colorectal cancer between 2001 and 2017. Demographic and clinical data, including gender, disease stage, age at diagnosis, recurrence status, and treatment details, were meticulously recorded. We rigorously evaluated various predictive models, including Decision Trees, Random Forests, Random Survival Forests (RSF), Gradient Boosting, mboost, Deep Learning Neural Network (DLNN), and Cox regression. Performance metrics, such as sensitivity, positive predictive value (PPV), specificity, area under the receiver operating characteristic curve (ROC area), and overall accuracy, were calculated for each model to predict mortality and disease recurrence. The analysis was performed using R version 4.1.3 software and the Python programming language.
For mortality prediction, the mboost model demonstrated the highest sensitivity at 96.9% (95% CI: 0.83-0.99) and an ROC area of 0.88. It also exhibited high specificity at 80% (95% CI: 0.59-0.93), a positive predictive value of 86.1% (95% CI: 0.70-0.95), and an overall accuracy of 89% (95% CI: 0.78-0.96). Random Forests showed perfect sensitivity of 100% (95% CI: 0.85-1) but had low specificity at 0% (95% CI: 0-0.52) and poor overall accuracy (50%). On the other hand, DLNN had the lowest performance metrics for mortality prediction, with a sensitivity of 24% (95% CI: 0.222-0.268), specificity of 75% (95% CI: 0.73-0.77), and a lower positive predictive value of 42% (95% CI: 0.38-0.45). The Gradient Boosting model showed the best performance in predicting recurrence, achieving perfect sensitivity of 100% (95% CI: 0.87-1) and high specificity at 92.9% (95% CI: 0.76-0.99). It also had a high positive predictive value of 93.3% (95% CI: 0.77-0.99). Gradient Boosting, with an ROC area of 96.4%, and mboost, with an ROC area of 75%, demonstrated remarkable performance. DLNN had the lowest performance metrics for recurrence prediction, with sensitivity at 1.75% (95% CI: 0.01-0.02), specificity at 98% (95% CI: 0.97-0.98), and a lower positive predictive value at 52.6% (95% CI: 0.39-0.65).
In summary, the mboost model demonstrated outstanding performance in predicting mortality, achieving exceptional results across various evaluation metrics. Random Forests exhibited perfect sensitivity but showed poor specificity and overall accuracy. The DLNN model displayed the lowest performance metrics for mortality prediction. In terms of recurrence prediction, the Gradient Boosting model outperformed other models with perfect sensitivity, high specificity, and positive predictive value. The DLNN model had the lowest performance metrics for recurrence prediction. Overall, the results emphasize the effectiveness of the mboost and Gradient Boosting models in predicting mortality and recurrence in colorectal cancer patients.
结直肠癌（CRC），也称为大肠癌，是一种死亡率很高的重大疾病，是全球第三大死亡原因。本研究的主要目的是评估预测模型在预测死亡事件和疾病复发概率方面的准确性。
对2001年至2017年间诊断为结直肠癌的284名患者进行回顾性分析。详细记录了人口统计学和临床数据，包括性别、疾病分期、诊断时年龄、复发状态和治疗细节。我们严格评估了各种预测模型，包括决策树、随机森林、随机生存森林（RSF）、梯度提升、mboost、深度学习神经网络（DLNN）和Cox回归。计算每个模型预测死亡率和疾病复发的性能指标，如敏感性、阳性预测值（PPV）、特异性、受试者操作特征曲线下面积（ROC面积）和总体准确性。使用R版本4.1.3软件和Python编程语言进行分析。
对于死亡率预测，mboost模型表现出最高的敏感性，为96.9%（95%CI：0.83-0.99），ROC面积为0.88。它还表现出高特异性，为80%（95%CI：0.59-0.93），阳性预测值为86.1%（95%CI：0.70-0.95），总体准确性为89%（95%CI：0.78-0.96）。随机森林显示出100%的完美敏感性（95%CI：0.85-1），但特异性低至0%（95%CI：0-0.52），总体准确性差（50%）。另一方面，DLNN在死亡率预测方面的性能指标最低，敏感性为24%（95%CI：0.222-0.268），特异性为75%（95%CI：0.73-0.77），阳性预测值较低，为42%（95%CI：0.38-0.45）。梯度提升模型在预测复发方面表现最佳，实现了100%的完美敏感性（95%CI：0.87-1）和92.9%的高特异性（95%CI：0.76-0.99）。它还具有93.3%的高阳性预测值（95%CI：0.77-0.99）。梯度提升的ROC面积为96.4%，mboost的ROC面积为75%，表现出色。DLNN在复发预测方面的性能指标最低，敏感性为1.75%（95%CI：0.01-0.02），特异性为98%（95%CI：0.97-0.98），阳性预测值较低，为52.6%（95%CI：0.39-0.65）。
总之，mboost模型在预测死亡率方面表现出色，在各种评估指标上都取得了优异的结果。随机森林表现出完美的敏感性，但特异性和总体准确性较差。DLNN模型在死亡率预测方面的性能指标最低。在复发预测方面，梯度提升模型优于其他模型，具有完美的敏感性、高特异性和阳性预测值。DLNN模型在复发预测方面的性能指标最低。总体而言，结果强调了mboost和梯度提升模型在预测结直肠癌患者死亡率和复发方面的有效性。
