Wang Xiaoqin, Welch Vivian, Li Meixuan, Yao Liang, Littell Julia, Li Huijuan, Yang Nan, Wang Jianjian, Shamseer Larissa, Chen Yaolong, Yang Kehu, Grimshaw Jeremy M
Evidence-based Medicine Centre of Lanzhou University Lanzhou China.
Clinical Epidemiology Program, Ottawa Hospital Research Institute Ottawa Ontario Canada.
Campbell Syst Rev. 2021 Feb 7;17(1):e1134. doi: 10.1002/cl2.1134. eCollection 2021 Mar.
The Campbell Collaboration undertakes systematic reviews of the effects of social and economic policies (interventions) to help policymakers, practitioners, and the public to make well-informed decisions about policy interventions. In 2010, the Cochrane Collaboration and the Campbell Collaboration developed a voluntary co-registration policy under the rationale to make full use of the shared interests and diverse expertise from different review groups within these two organizations. In order to promote the methodological quality and transparency of Campbell intervention reviews, the Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) were introduced in 2014 to guide Campbell reviewers. However, there has not been a comprehensive review of the methodological quality and reporting characteristics of Campbell reviews.
This review aimed to assess the methodological and reporting characteristics of Campbell intervention reviews and to compare the methodological quality and reporting completeness of Campbell reviews published before and after the implementation of MECCIR. A secondary aim was to compare the methodological quality and reporting completeness of reviews registered with Campbell only versus those co-registered with Cochrane and Campbell.
We searched the Campbell Library to identify all the completed intervention reviews published between 1 January 2011 to 31 January 2018.
One researcher downloaded and screened all the records to exclude non-intervention reviews based on reviews' title and abstract. A second researcher checked the full text of all the excluded records to confirm the exclusion. In case of discrepancies, the two researchers jointly agreed on the final decision.
We developed the abstraction form based on mandatory reporting items for methods, results, and discussion from the MECCIR reporting standards Version 1.1; and additional epidemiological characteristics identified in a similar study of systematic reviews in health. Additionally, we judged the methodological quality and completeness of reporting of each included review. For methodological quality, we used the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) instrument; for reporting completeness we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. We rated reporting as either complete/partial or not reported. We described characteristics of the included reviews with frequencies and percentages, and median with interquartile ranges (IQRs). We used Stata version 12.0 to conduct multiple linear regressions for continuous data and the ordered logistic regressions for ordered data to investigate associations between prespecified factors and both methodological quality and completeness of reporting.
We included 96 Campbell reviews, 46 were published between January 2011 and September 2014 (pre-MECCIR) and 50 between October 2014 and January 2018 (post-MECCIR). Twenty-two of 96 (23%) reviews were co-registered with Cochrane. For overall methodological quality, 16 (17%) reviews were rated as high, 40 (42%) as moderate, 24 (25%) as low and 16 (17%) as critical low using AMSTAR 2. Reviews published after the release of MECCIR had better methodological quality ratings than those published before MECCIR (odds ratio [OR] =6.61, 95% confidence interval [CI] [2.86, 15.27], < .001). The percentages of reviews of high or moderate quality were 76% (post-MECCIR) and 39% (pre-MECCIR). Reviews co-registered with Cochrane were rated as having better methodological quality than those registered only with Campbell (OR = 5.57, 95% CI [2.13, 14.58], < .001). The percentages of reviews of high or moderate quality were 77% versus 53% between co-registered and Campbell registered only reviews. Twenty-five of 96 reviews (26%) completely or partially reported all 27 PRISMA checklist items. The median number of items reported across reviews was 25 (IQR, 22-26). Reviews published after the release of MECCIR reported 2.80 more items than those published before MECCIR (95% CI [1.74, 3.88], < .001); reviews co-registered on Campbell and Cochrane reported 1.98 more items than reviews only registered in Campbell (95% CI [0.72, 3.24], = .003). An increasing trend over time was observed for both the percentage of high and moderate methodological quality of reviews and the median number of PRISMA items reported.
AUTHORS' CONCLUSIONS: Many features expected in systematic reviews were present in Campbell reviews most of the time. Methodological quality and reporting completeness were both significantly higher in reviews published after the introduction of MECCIR in 2014 compared with those published before. However, this may also reflect general improvement in the reporting the methodology of systematic reviews over time or associations with other characteristics which were not assessed such as funding or experience of teams. Reviews co-registered with Cochrane were of higher methodological quality and more complete reporting than reviews only registered in Campbell.
坎贝尔协作组织对社会和经济政策（干预措施）的效果进行系统评价，以帮助政策制定者、从业者和公众就政策干预做出明智决策。2010年，考克兰协作组织和坎贝尔协作组织制定了一项自愿联合注册政策，其依据是充分利用这两个组织内不同评价小组的共同利益和多样的专业知识。为提高坎贝尔干预评价的方法学质量和透明度，2014年引入了《坎贝尔协作组织干预评价方法学期望》（MECCIR）来指导坎贝尔评价人员。然而，尚未对坎贝尔评价的方法学质量和报告特征进行全面综述。
本综述旨在评估坎贝尔干预评价的方法学和报告特征，并比较MECCIR实施前后发表的坎贝尔评价的方法学质量和报告完整性。次要目的是比较仅在坎贝尔注册的评价与在考克兰和坎贝尔联合注册的评价的方法学质量和报告完整性。
我们检索了坎贝尔图书馆，以识别2011年1月1日至2018年1月31日期间发表的所有已完成的干预评价。
一名研究人员根据评价的标题和摘要下载并筛选所有记录，以排除非干预评价。第二名研究人员检查所有排除记录的全文以确认排除。如有分歧，两名研究人员共同商定最终决定。
我们根据MECCIR报告标准1.1版中关于方法、结果和讨论的强制报告项目开发了提取表；以及在一项类似的卫生系统评价研究中确定的其他流行病学特征。此外，我们判断了每项纳入评价的方法学质量和报告完整性。对于方法学质量，我们使用AMSTAR 2（评估系统评价的测量工具2）工具；对于报告完整性，我们使用PRISMA（系统评价和Meta分析的首选报告项目）清单。我们将报告评为完整/部分或未报告。我们用频率和百分比描述纳入评价的特征，用中位数和四分位数间距（IQR）描述。我们使用Stata 12.0版对连续数据进行多元线性回归，对有序数据进行有序逻辑回归，以研究预先设定的因素与方法学质量和报告完整性之间的关联。
我们纳入了96项坎贝尔评价，其中46项在2011年1月至2014年9月（MECCIR之前）发表，50项在2014年10月至2018年1月（MECCIR之后）发表。96项评价中有22项（23%）与考克兰联合注册。对于总体方法学质量，使用AMSTAR 2评估，16项（17%）评价被评为高质量，40项（42%）为中等质量，24项（25%）为低质量，16项（17%）为极低质量。MECCIR发布后发表的评价比MECCIR发布前发表的评价具有更好的方法学质量评级（优势比[OR]=6.61，95%置信区间[CI][2.86,15.27]，P<0.001）。高质量或中等质量评价的百分比在MECCIR之后为76%，在MECCIR之前为39%。与仅在坎贝尔注册的评价相比，与考克兰联合注册的评价被评为具有更好的方法学质量（OR=5.57，95%CI[2.13,14.58]，P<0.001）。联合注册和仅在坎贝尔注册的评价中，高质量或中等质量评价的百分比分别为77%和53%。96项评价中有25项（26%）完全或部分报告了PRISMA清单的所有27项。各评价报告项目的中位数为25项（IQR，22 - 26）。MECCIR发布后发表的评价比MECCIR发布前发表的评价多报告2.80项（95%CI[1.74,3.88]，P<0.001）；在坎贝尔和考克兰联合注册的评价比仅在坎贝尔注册的评价多报告1.98项（95%CI[0.72,3.24]，P = 0.003）。随着时间的推移，评价的高质量和中等质量百分比以及报告的PRISMA项目中位数均呈现上升趋势。
系统评价中预期的许多特征在大多数时候的坎贝尔评价中都存在。与2014年引入MECCIR之前发表的评价相比，2014年引入MECCIR之后发表的评价在方法学质量和报告完整性方面均显著更高。然而，这也可能反映出随着时间的推移，系统评价方法学报告的总体改进，或者与未评估的其他特征（如资金或团队经验）的关联。与仅在坎贝尔注册的评价相比，与考克兰联合注册的评价具有更高的方法学质量和更完整的报告。
