Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada.
PLoS One. 2011 Feb 24;6(2):e17242. doi: 10.1371/journal.pone.0017242.
The Risk of Bias (RoB) tool is used to assess internal validity of randomized controlled trials (RCTs). Our objectives were to: 1) evaluate inter-rater agreement of the RoB tool; 2) determine the time to access supplemental study information; 3) compare the RoB tool with the Jadad scale and Schulz allocation concealment (AC); and 4) examine the relationship between RoB and effect estimates.
We conducted a systematic review of long-acting beta agonists (LABA) combined with inhaled corticosteroids (ICS) for adults with persistent asthma. Two reviewers independently assessed 107 trials using RoB, Jadad, and AC. One reviewer searched for study protocols. We assessed inter-rater agreement using weighted Kappa (κ) and the correlation between tools using Kendall's Tau (τ). Mean differences in effect sizes for RCTs with different RoB were calculated using inverse variance method and random effects model.
Trials had good Jadad scores (median 4, IQR 3-4); however, 85% had unclear AC and 87% high RoB. The factor that most influenced RoB was the potential inappropriate influence of study sponsors (95% industry funded). Agreement on RoB domains was fair (κ = 0.40) to almost perfect (κ = 0.86), and moderate for overall RoB (κ = 0.41). Median time to complete RoB assessments was 21 minutes (IQR 14-27) and 12 minutes (IQR 9-16) to search for protocols. Protocols were identified for 5/42 studies (12%); in 3 cases the assessment of selective outcome reporting changed. There was low correlation between overall RoB vs. Jadad (τ = 0.04, p = 0.3) and AC (τ = -0.02, p = 0.7). Analyses comparing effect estimates and risk showed no important patterns.
Inter-rater agreement on RoB assessments was better than previously reported suggesting that review-specific guidelines are important. The correlation between RoB and Jadad was low suggesting measurement of different constructs (risk of bias vs. quality of reporting). The extensive involvement of the pharmaceutical industry in this LABA/ICS research should raise concerns about potential overestimates of treatment effects.
风险偏倚 (RoB) 工具用于评估随机对照试验 (RCT) 的内部有效性。我们的目标是：1）评估 RoB 工具的评分者间一致性；2）确定获取补充研究信息的时间；3）比较 RoB 工具与 Jadad 量表和 Schulz 分配隐藏 (AC)；4）研究 RoB 与效应估计之间的关系。
我们对长效β激动剂 (LABA) 联合吸入皮质类固醇 (ICS) 治疗持续性哮喘的成人进行了系统评价。两位审稿人独立使用 RoB、Jadad 和 AC 对 107 项试验进行评估。一位审稿人搜索研究方案。我们使用加权 Kappa (κ) 评估评分者间一致性，并使用 Kendall's Tau (τ) 评估工具间相关性。使用Inverse Variance 法和随机效应模型计算不同 RoB 的 RCT 效应大小的均值差异。
试验的 Jadad 评分较高（中位数 4 分，IQR 3-4 分）；然而，85%的试验 AC 不明确，87%的试验 RoB 较高。影响 RoB 的最重要因素是研究赞助商的潜在不适当影响（95%为行业资助）。RoB 各领域的一致性为中等至高（κ=0.40-0.86），总体 RoB 的一致性为中等（κ=0.41）。完成 RoB 评估的中位数时间为 21 分钟（IQR 14-27），搜索方案的中位数时间为 12 分钟（IQR 9-16）。确定了 5/42 项研究（12%）的方案；在 3 项研究中，选择性结果报告的评估发生了变化。总体 RoB 与 Jadad（τ=0.04，p=0.3）和 AC（τ=0.02，p=0.7）之间的相关性较低。比较效应估计值和风险的分析没有显示出重要的模式。
RoB 评估的评分者间一致性优于之前的报告，表明特定于综述的指南很重要。RoB 与 Jadad 的相关性较低，表明测量的是不同的结构（偏倚风险与报告质量）。制药行业在这项 LABA/ICS 研究中的广泛参与应该引起对治疗效果高估的担忧。
