Ban Jong-Wook, Emparanza José Ignacio, Urreta Iratxe, Burls Amanda
Evidence-Based Health Care Programme, Department of Continuing Education, Kellogg College, University of Oxford, Oxford, United Kingdom.
CASPe, CIBER-ESP, Clinical Epidemiology Unit, Hospital Universitario Donostia, San Sebastian, Spain.
PLoS One. 2016 Jan 5;11(1):e0145779. doi: 10.1371/journal.pone.0145779. eCollection 2016.
Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules' performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics.
Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described.
A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2-4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively.
Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved.
许多新的临床预测规则被推导和验证。但临床预测研究的设计和报告质量一直不尽如人意。我们旨在评估验证研究的设计特征是否与临床预测规则性能的高估有关。我们还旨在评估验证研究是否清晰报告了重要的方法学特征。
检索电子数据库，查找2006年至2010年发表的临床预测规则研究的系统评价。从系统评价中纳入的合格验证研究中提取数据。采用meta分析的meta流行病学方法评估设计特征对预测性能的影响。从每项验证研究中，评估7个设计特征和7个报告特征是否得到恰当描述。
从15项系统评价（31项meta分析）中收集了总共287项临床预测规则的验证研究。采用病例对照设计的验证研究得出的汇总诊断比值比（DOR）比采用队列设计和设计不明确的验证研究大2.2倍（95%CI：1.2 - 4.3）。与完全、部分和不明确验证相比，采用差异验证时，汇总DOR被高估了两倍（95%CI：1.2 - 3.1）。样本量不足的验证研究的汇总相对诊断比值比（RDOR）为1.9（95%CI：1.2 - 3.1），而样本量充足的研究为1.9（95%CI：1.2 - 3.1）。分别有10.1%、9.4%和7.0%的验证研究充分描述了研究地点、可靠性和临床预测规则。
存在设计缺陷的验证研究可能高估临床预测规则的性能。验证临床预测规则的研究的报告质量需要提高。
