Ban Jong-Wook, Stevens Richard, Perera Rafael
1Evidence-Based Health Care Programme, Centre for Evidence-Based Medicine, University of Oxford, Oxford, UK.
2Nuffield Department of Primary Care Health Sciences, Medical Science Division, University of Oxford, Oxford, UK.
Diagn Progn Res. 2018 Feb 6;2:3. doi: 10.1186/s41512-018-0025-6. eCollection 2018.
Clinical prediction rules (CPRs) should be externally validated by independent researchers. Although there are many cardiovascular CPRs, most have not been externally validated. It is not known why some CPRs are externally validated by independent researchers and others are not.
We analyzed cardiovascular risk CPRs included in a systematic review. Independent external validations were identified by forward citation searches of derivation studies. Time between the publication of a cardiovascular CPR and the first independent external validation was calculated. We assessed Kaplan-Meier estimates of the probability to have an independent external validation. Using univariable Cox regression, we explored whether characteristics of derivation (design, location, sample size, number of predictors, presentation format, validation in derivation), reporting (participants, predictors, outcomes, performance measure, information for risk calculation), and publication (journal impact factor) are associated with time to the first independent external validation.
Of 125 cardiovascular risk CPRs, 29 had an independent external validation. The median follow-up was 118 months (95% CI, 99-130). The 25th percentile of event time was 122 months (95% CI, 91-299). Cardiovascular risk CPRs from the USA were 4.15 times (95% CI, 1.89-9.13) more likely to have an independent external validation. Increasing the sample size of derivation by ten times was associated with a 2.32-fold (95% CI, 1.37-3.91) increase in the probability of having an independent external validation. CPRs presented with an internal validation tend to get an independent external validation sooner (HR = 1.73, 95% CI, 0.77-3.93). CPRs reporting all the information necessary for calculating individual risk were 2.65 (95% CI, 1.01-6.96) times more likely to have an independent external validation. Publishing a cardiovascular risk CPR in a journal that has one unit higher impact factor was associated with a 6% (95% CI, 3-9) higher likelihood of an independent external validation.
The probability for cardiovascular risk CPRs to get an independent external validation was low even many years after their derivations. Authors of new cardiovascular risk CPRs should consider using adequate sample size, conducting an internal validation, and reporting all the information needed for risk calculation as these features were associated with an independent external validation.
临床预测规则（CPRs）应由独立研究人员进行外部验证。虽然有许多心血管疾病的CPRs，但大多数尚未经过外部验证。目前尚不清楚为何有些CPRs能被独立研究人员进行外部验证，而其他的则没有。
我们分析了一项系统评价中纳入的心血管疾病风险CPRs。通过对推导研究的正向引文检索来确定独立的外部验证。计算心血管疾病CPR发表与首次独立外部验证之间的时间。我们评估了进行独立外部验证概率的Kaplan-Meier估计值。使用单变量Cox回归，我们探讨了推导（设计、地点、样本量、预测因子数量、呈现形式、推导中的验证）、报告（参与者、预测因子、结局、性能指标、风险计算信息）和发表（期刊影响因子）的特征是否与首次独立外部验证的时间相关。
在125个心血管疾病风险CPRs中，29个有独立的外部验证。中位随访时间为118个月（95%CI，99 - 130）。事件时间的第25百分位数为122个月（95%CI，91 - 299）。来自美国的心血管疾病风险CPRs进行独立外部验证的可能性高4.15倍（95%CI，1.89 - 9.13）。将推导样本量增加10倍与进行独立外部验证概率增加2.32倍（95%CI，1.37 - 3.91）相关。进行了内部验证的CPRs往往能更快获得独立外部验证（HR = 1.73，95%CI，0.77 - 3.93）。报告了计算个体风险所需所有信息的CPRs进行独立外部验证的可能性高2.65倍（95%CI，1.01 - 6.96）。在影响因子高一个单位的期刊上发表心血管疾病风险CPR与独立外部验证可能性高6%（95%CI，3 - 9）相关。
即使在推导多年后，心血管疾病风险CPRs获得独立外部验证的概率仍然很低。新的心血管疾病风险CPRs的作者应考虑使用足够的样本量、进行内部验证并报告风险计算所需的所有信息，因为这些特征与独立外部验证相关。
