使用PRISMA-A评估有荟萃分析的系统评价摘要的报告质量以及无经验评分者之间评估的不一致性。
Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience.
作者信息
Maticic Katarina, Krnic Martinic Marina, Puljak Livia
机构信息
Pharmacy Mirna Jelic Hrgic, Brodski Stupnik, Croatia.
Department of ENT, Head and Neck Surgery, University Hospital Center, Split, Croatia.
出版信息
BMC Med Res Methodol. 2019 Feb 14;19(1):32. doi: 10.1186/s12874-019-0675-2.
BACKGROUND
Reporting quality of systematic reviews' (SRs) abstracts is important because this is often the only information about a study that readers have. The aim of this study was to assess adherence of SR abstracts in the field of anesthesiology with the reporting checklist PRISMA extension for Abstracts (PRISMA-A) and to analyze to what extent will the use of PRISMA-A yield concordant ratings in two raters without prior experience with the checklist.
METHODS
We analyzed reporting quality of SRs with meta-analysis of randomized controlled trials of interventions published in the field of anesthesiology from 2012 to 2016 by using 12-item PRISMA-A checklist. After calibration exercise, two authors without prior experience with PRISMA-A scored the abstracts. Primary outcome was median adherence to PRISMA-A checklist. Secondary outcome was adherence to individual items of the checklist. We analyzed whether there was improvement in reporting of SR abstracts over time. Additionally, we analyzed discrepancies between the two raters in scoring individual PRISMA-A items.
RESULTS
Our search yielded 318 results, of which we included 244 SRs. Median adherence to PRISMA-A checklist was 42% (5 items of 12). The majority of analyzed SR abstracts (N = 148, 61%) had a total adherence score under 50%, and not a single one had adherence above 75%. Adherence to individual items was very variable, ranging from 0% for reporting SR funding, to 97% for interpreting SR findings. Overall adherence to PRISMA-A did not change over the analyzed 5 years before and after publication of PRISMA-A in 2013. Even after calibration exercise, discrepancies between the two raters were found in 275 (9.3%) out of 2928 analyzed PRISMA-A items. Cohen's Kappa was 0.807. In the item about the description of effect there were discrepancies in 59% of the abstracts between the raters.
CONCLUSION
Reporting quality of systematic review abstracts in the field of anesthesiology is suboptimal, and did not improve after publication of PRISMA-A checklist in 2013. We need stricter adherence to reporting checklists by authors, editors and peer-reviewers, and interventions that will help those stakeholders to improve reporting of systematic reviews. Some items of PRISMA-A checklist are difficult to score.
背景
系统评价（SR）摘要的报告质量很重要，因为这通常是读者所拥有的关于一项研究的唯一信息。本研究的目的是评估麻醉学领域SR摘要对摘要报告清单PRISMA扩展版（PRISMA-A）的依从性，并分析使用PRISMA-A在两名无该清单使用经验的评分者中产生一致评分的程度。
方法
我们使用12项PRISMA-A清单分析了2012年至2016年发表在麻醉学领域的干预性随机对照试验的SR报告质量。经过校准练习后，两名无PRISMA-A使用经验的作者对摘要进行评分。主要结局是对PRISMA-A清单的中位依从性。次要结局是对清单各个条目的依从性。我们分析了SR摘要的报告随时间是否有改善。此外，我们分析了两名评分者在对PRISMA-A各个条目评分时的差异。
结果
我们的检索产生了318条结果，其中我们纳入了244篇SR。对PRISMA-A清单的中位依从性为42%（12项中的5项）。大多数分析的SR摘要（N = 148，61%）的总依从性得分低于50%，没有一篇的依从性高于75%。对各个条目的依从性差异很大，从报告SR资金的0%到解释SR结果的97%不等。在2013年PRISMA-A发布前后的5年分析期内，对PRISMA-A的总体依从性没有变化。即使经过校准练习，在2928个分析的PRISMA-A条目中，两名评分者之间仍发现275个（9.3%）存在差异。Cohen's Kappa为0.807。在关于效应描述的条目中，评分者之间在59%的摘要中存在差异。
结论
麻醉学领域系统评价摘要的报告质量欠佳，在2013年PRISMA-A清单发布后并未改善。我们需要作者、编辑和同行评审者更严格地遵守报告清单，以及采取有助于这些利益相关者改善系统评价报告的干预措施。PRISMA-A清单的一些条目难以评分。
Zotero 插件