Nattino Giovanni, Finazzi Stefano, Bertolini Guido
GiViTI Coordinating Center, Laboratory of Clinical Epidemiology, IRCCS - Istituto di Ricerche Farmacologiche 'Mario Negri', Villa Camozzi, Ranica (BG), Italy.
Laboratoire Matériaux et Phénomènes Quantiques, Université Paris Diderot-Paris 7, Paris, France.
Stat Med. 2016 Feb 28;35(5):709-20. doi: 10.1002/sim.6744. Epub 2015 Oct 5.
A prognostic model is well calibrated when it accurately predicts event rates. This is first determined by testing for goodness of fit with the development dataset. All existing tests and graphic tools designed for the purpose suffer several drawbacks, related mainly to the subgrouping of observations or to heavy dependence on arbitrary parameters. We propose a statistical test and a graphical method to assess the goodness of fit of logistic regression models, obtained through an extension of similar techniques developed for external validation. We analytically computed and numerically verified the distribution of the underlying statistic. Simulations on a set of realistic scenarios show that this test and the well-known Hosmer-Lemeshow approach have similar type I error rates. The main advantage of this new approach is that the relationship between model predictions and outcome rates across the range of probabilities can be represented in the calibration belt plot, together with its statistical confidence. By readily spotting any deviations from the perfect fit, this new graphical tool is designed to identify, during the process of model development, poorly modeled variables that call for further investigation. This is illustrated through an example based on real data.
当预后模型能够准确预测事件发生率时，它就得到了良好的校准。这首先要通过对开发数据集进行拟合优度检验来确定。为此目的设计的所有现有检验和图形工具都存在几个缺点，主要与观测值的分组或对任意参数的严重依赖有关。我们提出了一种统计检验和一种图形方法，以评估逻辑回归模型的拟合优度，这是通过扩展为外部验证而开发的类似技术获得的。我们对基础统计量的分布进行了解析计算和数值验证。在一组现实场景上的模拟表明，该检验与著名的霍斯默 - 莱梅肖方法具有相似的第一类错误率。这种新方法的主要优点是，在校准带图中可以表示模型预测与概率范围内的结果率之间的关系及其统计置信度。通过容易地发现与完美拟合的任何偏差，这种新的图形工具旨在在模型开发过程中识别需要进一步研究的建模不佳的变量。这通过一个基于实际数据的例子进行了说明。
