Tournoud Maud, Larue Audrey, Cazalis Marie-Angelique, Venet Fabienne, Pachot Alexandre, Monneret Guillaume, Lepape Alain, Veyrieras Jean-Baptiste
Bioinformatics Research Department, bioMérieux, Marcy L'Etoile, France.
Medical Diagnostic Discovery Department, bioMérieux, Marcy L'Etoile, France.
BMC Bioinformatics. 2015 Mar 28;16:106. doi: 10.1186/s12859-015-0537-9.
Construction and validation of a prognostic model for survival data in the clinical domain is still an active field of research. Nevertheless there is no consensus on how to develop routine prognostic tests based on a combination of RT-qPCR biomarkers and clinical or demographic variables. In particular, the estimation of the model performance requires to properly account for the RT-qPCR experimental design.
We present a strategy to build, select, and validate a prognostic model for survival data based on a combination of RT-qPCR biomarkers and clinical or demographic data and we provide an illustration on a real clinical dataset. First, we compare two cross-validation schemes: a classical outcome-stratified cross-validation scheme and an alternative one that accounts for the RT-qPCR plate design, especially when samples are processed by batches. The latter is intended to limit the performance discrepancies, also called the validation surprise, between the training and the test sets. Second, strategies for model building (covariate selection, functional relationship modeling, and statistical model) as well as performance indicators estimation are presented. Since in practice several prognostic models can exhibit similar performances, complementary criteria for model selection are discussed: the stability of the selected variables, the model optimism, and the impact of the omitted variables on the model performance.
On the training dataset, appropriate resampling methods are expected to prevent from any upward biases due to unaccounted technical and biological variability that may arise from the experimental and intrinsic design of the RT-qPCR assay. Moreover, the stability of the selected variables, the model optimism, and the impact of the omitted variables on the model performances are pivotal indicators to select the optimal model to be validated on the test dataset.
临床领域生存数据预后模型的构建与验证仍是一个活跃的研究领域。然而，对于如何基于逆转录定量聚合酶链反应（RT-qPCR）生物标志物与临床或人口统计学变量的组合来开发常规预后测试，尚无共识。特别是，模型性能的估计需要适当考虑RT-qPCR实验设计。
我们提出了一种基于RT-qPCR生物标志物与临床或人口统计学数据的组合来构建、选择和验证生存数据预后模型的策略，并在一个真实的临床数据集上进行了说明。首先，我们比较了两种交叉验证方案：经典的结果分层交叉验证方案和另一种考虑RT-qPCR板设计的方案，特别是当样本分批处理时。后者旨在限制训练集和测试集之间的性能差异，也称为验证意外。其次，介绍了模型构建策略（协变量选择、功能关系建模和统计模型）以及性能指标估计。由于在实践中几个预后模型可能表现出相似的性能，因此讨论了模型选择的补充标准：所选变量的稳定性、模型乐观性以及遗漏变量对模型性能的影响。
在训练数据集上，预期适当的重采样方法可防止因RT-qPCR检测的实验和内在设计可能产生的未考虑的技术和生物学变异性而导致的任何向上偏差。此外，所选变量的稳定性、模型乐观性以及遗漏变量对模型性能的影响是选择要在测试数据集上验证的最佳模型的关键指标。
