在评估包括随机和非随机研究的系统综述时，AMSTAR 2 与 ROBIS 之间存在细微差异。

Minor differences were found between AMSTAR 2 and ROBIS in the assessment of systematic reviews including both randomized and nonrandomized studies.

机构信息

Institute for Research in Operative Medicine, Evidence-based Health Services Research, Faculty of Health, School of Medicine, Witten/Herdecke University, Ostmerheimer Str, Cologne 200 51109, Germany.

Catholic University of Croatia, Ilica 242, Zagreb 10000, Croatia.

出版信息

J Clin Epidemiol. 2019 Apr;108:26-33. doi: 10.1016/j.jclinepi.2018.12.004. Epub 2018 Dec 10.

DOI:10.1016/j.jclinepi.2018.12.004

PMID:30543911

Abstract

OBJECTIVE

To compare A Measurement Tool to Assess Systematic Reviews (AMSTAR 2) with a tool to assess risk of bias in systematic reviews (ROBIS) in terms of validity, reliability, and applicability.

STUDY DESIGN AND SETTING

We analyzed 30 systematic reviews (SRs) that included randomized and nonrandomized studies, with Cochrane and non-Cochrane SRs sampled in 1:1 ratio. Four reviewers assessed independently all 30 SRs with AMSTAR 2, followed by ROBIS. We calculated Fleiss' Kappa as a measure of inter-rater reliability (IRR) across 4 raters.

RESULTS

The IRR for scoring the overall confidence in the SRs with AMSTAR 2 and the overall domain in ROBIS was fair (AMSTAR 2: κ = 0.30, 95% [confidence interval] CI: 0.17 to 0.43; ROBIS: κ = 0.28, 95% CI: 0.13 to 0.42). AMSTAR 2 confidence in review ratings strongly correlated with the overall domain rating in ROBIS (Spearman r = 0.84). Mean time for scoring AMSTAR 2 was slightly higher than for ROBIS (18 vs. 16 min), with huge differences between the reviewers.

CONCLUSION

Both AMSTAR 2 and ROBIS can be applied to SRs including both randomized controlled trials (RCTs) and non-RCTs. Measurement properties of ROBIS seemed not to be much different when comparing with other studies that include only SRs of RCTs.

摘要

目的

比较评估系统评价的工具（AMSTAR 2）和评估系统评价偏倚风险的工具（ROBIS）在有效性、可靠性和适用性方面的差异。

研究设计与设置

我们分析了 30 篇系统评价（SRs），其中包括随机和非随机研究，以 1:1 的比例抽样 Cochrane 和非 Cochrane SRs。四位评审员独立使用 AMSTAR 2 对所有 30 篇 SRs 进行评估，然后使用 ROBIS。我们计算了 Fleiss' Kappa 作为 4 位评审员之间的评分者间信度（IRR）的衡量标准。

结果

使用 AMSTAR 2 对 SRs 的整体可信度和 ROBIS 的整体领域进行评分的 IRR 为中等（AMSTAR 2：κ=0.30，95%置信区间[CI]：0.17 至 0.43；ROBIS：κ=0.28，95%CI：0.13 至 0.42）。AMSTAR 2 对评价等级的可信度与 ROBIS 的整体领域等级高度相关（Spearman r=0.84）。使用 AMSTAR 2 进行评分的平均时间略高于 ROBIS（18 分钟与 16 分钟），但评审员之间的差异巨大。