Goldstein Benjamin A, Polley Eric C, Briggs Farren B S, van der Laan Mark J, Hubbard Alan
Int J Biostat. 2016 May 1;12(1):117-29. doi: 10.1515/ijb-2015-0014.
Comparing the relative fit of competing models can be used to address many different scientific questions. In classical statistics one can, if appropriate, use likelihood ratio tests and information based criterion, whereas clinical medicine has tended to rely on comparisons of fit metrics like C-statistics. However, for many data adaptive modelling procedures such approaches are not suitable. In these cases, statisticians have used cross-validation, which can make inference challenging. In this paper we propose a general approach that focuses on the "conditional" risk difference (conditional on the model fits being fixed) for the improvement in prediction risk. Specifically, we derive a Wald-type test statistic and associated confidence intervals for cross-validated test sets utilizing the independent validation within cross-validation in conjunction with a test for multiple comparisons. We show that this test maintains proper Type I Error under the null fit, and can be used as a general test of relative fit for any semi-parametric model alternative. We apply the test to a candidate gene study to test for the association of a set of genes in a genetic pathway.
比较竞争模型的相对拟合优度可用于解决许多不同的科学问题。在经典统计学中，如果合适的话，可以使用似然比检验和基于信息的准则，而临床医学往往依赖于拟合指标（如C统计量）的比较。然而，对于许多数据自适应建模过程，这些方法并不适用。在这些情况下，统计学家使用交叉验证，这可能会使推断具有挑战性。在本文中，我们提出了一种通用方法，该方法侧重于预测风险改善的“条件”风险差异（以模型拟合固定为条件）。具体而言，我们利用交叉验证中的独立验证结合多重比较检验，为交叉验证测试集推导了一个Wald型检验统计量和相关的置信区间。我们表明，该检验在零拟合下保持适当的I型错误，并且可以用作任何半参数模型替代方案的相对拟合优度的通用检验。我们将该检验应用于一项候选基因研究，以测试一组基因在遗传途径中的关联性。
