Cevenini Gabriele, Barbini Emanuela, Scolletta Sabino, Biagioli Bonizella, Giomarelli Pierpaolo, Barbini Paolo
Department of Surgery and Bioengineering, University of Siena, Siena, Italy.
BMC Med Inform Decis Mak. 2007 Nov 22;7:36. doi: 10.1186/1472-6947-7-36.
Popular predictive models for estimating morbidity probability after heart surgery are compared critically in a unitary framework. The study is divided into two parts. In the first part modelling techniques and intrinsic strengths and weaknesses of different approaches were discussed from a theoretical point of view. In this second part the performances of the same models are evaluated in an illustrative example.
Eight models were developed: Bayes linear and quadratic models, k-nearest neighbour model, logistic regression model, Higgins and direct scoring systems and two feed-forward artificial neural networks with one and two layers. Cardiovascular, respiratory, neurological, renal, infectious and hemorrhagic complications were defined as morbidity. Training and testing sets each of 545 cases were used. The optimal set of predictors was chosen among a collection of 78 preoperative, intraoperative and postoperative variables by a stepwise procedure. Discrimination and calibration were evaluated by the area under the receiver operating characteristic curve and Hosmer-Lemeshow goodness-of-fit test, respectively.
Scoring systems and the logistic regression model required the largest set of predictors, while Bayesian and k-nearest neighbour models were much more parsimonious. In testing data, all models showed acceptable discrimination capacities, however the Bayes quadratic model, using only three predictors, provided the best performance. All models showed satisfactory generalization ability: again the Bayes quadratic model exhibited the best generalization, while artificial neural networks and scoring systems gave the worst results. Finally, poor calibration was obtained when using scoring systems, k-nearest neighbour model and artificial neural networks, while Bayes (after recalibration) and logistic regression models gave adequate results.
Although all the predictive models showed acceptable discrimination performance in the example considered, the Bayes and logistic regression models seemed better than the others, because they also had good generalization and calibration. The Bayes quadratic model seemed to be a convincing alternative to the much more usual Bayes linear and logistic regression models. It showed its capacity to identify a minimum core of predictors generally recognized as essential to pragmatically evaluate the risk of developing morbidity after heart surgery.
在一个统一的框架内对用于估计心脏手术后发病概率的常用预测模型进行了严格比较。该研究分为两个部分。在第一部分中，从理论角度讨论了不同方法的建模技术以及内在的优缺点。在第二部分中，通过一个示例对相同模型的性能进行了评估。
开发了八个模型：贝叶斯线性和二次模型、k近邻模型、逻辑回归模型、希金斯评分系统和直接评分系统，以及两个具有一层和两层的前馈人工神经网络。将心血管、呼吸、神经、肾脏、感染和出血性并发症定义为发病情况。使用了每组545例病例的训练集和测试集。通过逐步程序从78个术前、术中和术后变量中选择最佳预测变量集。分别通过受试者操作特征曲线下面积和 Hosmer-Lemeshow 拟合优度检验评估区分能力和校准情况。
评分系统和逻辑回归模型需要最大的预测变量集，而贝叶斯模型和k近邻模型则更为简约。在测试数据中，所有模型都表现出可接受的区分能力，然而仅使用三个预测变量的贝叶斯二次模型表现最佳。所有模型都表现出令人满意的泛化能力：同样，贝叶斯二次模型表现出最佳泛化能力，而人工神经网络和评分系统给出的结果最差。最后，使用评分系统、k近邻模型和人工神经网络时校准效果较差，而贝叶斯模型（重新校准后）和逻辑回归模型给出了合适的结果。
尽管在本示例中所有预测模型都表现出可接受的区分性能，但贝叶斯模型和逻辑回归模型似乎比其他模型更好，因为它们也具有良好的泛化和校准能力。贝叶斯二次模型似乎是比更为常用的贝叶斯线性模型和逻辑回归模型更有说服力的替代方案。它显示出能够识别出通常被认为对于实际评估心脏手术后发病风险至关重要的最小核心预测变量集的能力。
