School of Chinese Medicine, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, Hong Kong.
The Jockey Club School of Public Health and Primary Care, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, Hong Kong.
BMC Med Res Methodol. 2021 Oct 30;21(1):237. doi: 10.1186/s12874-021-01437-0.
While well-conducted systematic reviews (SRs) can provide the best evidence on the potential effectiveness of acupuncture, limitations on the methodological rigour of SRs may impact the trustworthiness of their conclusions. This cross-sectional study aimed to evaluate the methodological quality of a representative sample of SRs on acupuncture effectiveness.
Cochrane Database of Systematic Reviews, MEDLINE, and EMBASE were searched for SRs focusing on the treatment effect of manual acupuncture or electro-acupuncture published during January 2018 and March 2020. Eligible SRs must contain at least one meta-analysis and be published in English language. Two independent reviewers extracted the bibliographical characteristics of the included SRs with a pre-designed questionnaire and appraised the methodological quality of the studies with the validated AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2). The associations between bibliographical characteristics and methodological quality ratings were explored using Kruskal-Wallis rank tests and Spearman's rank correlation coefficients.
A total of 106 SRs were appraised. Only one (0.9%) SR was of high overall methodological quality, zero (0%) was of moderate-quality, six (5.7%) and 99 (93.4%) were of low-quality and critically low-quality respectively. Among appraised SRs, only ten (9.4%) provided an a priori protocol, four (3.8%) conducted a comprehensive literature search, five (4.7%) provided a list of excluded studies, and six (5.7%) performed meta-analysis appropriately. Cochrane SRs, updated SRs, and SRs that did not search non-English databases had relatively higher overall quality.
Methodological quality of SRs on acupuncture is unsatisfactory. Future reviewers should improve critical methodological aspects of publishing protocols, performing comprehensive search, providing a list of excluded studies with justifications for exclusion, and conducting appropriate meta-analyses. These recommendations can be implemented via enhancing the technical competency of reviewers in SR methodology through established education approaches as well as quality gatekeeping by journal editors and reviewers. Finally, for evidence users, skills in SR critical appraisal remain to be essential as relevant evidence may not be available in pre-appraised formats.
虽然精心进行的系统评价（SR）可以提供有关针灸潜在有效性的最佳证据，但 SR 方法学严谨性的限制可能会影响其结论的可信度。本横断面研究旨在评估一组具有代表性的关于针灸有效性的 SR 的方法学质量。
检索了 Cochrane 系统评价数据库、MEDLINE 和 EMBASE，以获取 2018 年 1 月至 2020 年 3 月发表的关于手动针灸或电针治疗效果的 SR。合格的 SR 必须包含至少一项荟萃分析，并以英文发表。两位独立的综述作者使用预先设计的问卷提取纳入的 SR 的文献特征，并使用经过验证的 AMSTAR 2（一种评估系统评价的测量工具）评估研究的方法学质量。使用 Kruskal-Wallis 秩检验和 Spearman 秩相关系数探索文献特征与方法学质量评分之间的关联。
共评价了 106 篇 SR。只有一篇（0.9%）SR 的整体方法学质量较高，一篇（0%）为中度质量，六篇（5.7%）和九十九篇（93.4%）为低质量和严重低质量。在评价的 SR 中，只有十篇（9.4%）提供了事先制定的方案，四篇（3.8%）进行了全面的文献检索，五篇（4.7%）提供了排除研究的清单，六篇（5.7%）进行了适当的荟萃分析。 Cochrane SR、更新的 SR 和未检索非英文数据库的 SR 的总体质量相对较高。
针灸 SR 的方法学质量不尽如人意。未来的评论员应改进发表方案的关键方法学方面，全面检索文献，提供排除研究的清单，并进行适当的荟萃分析。这些建议可以通过通过既定的教育方法提高评论员在 SR 方法学方面的技术能力，以及期刊编辑和评论员的质量把关来实现。最后，对于证据使用者来说，SR 批判性评价的技能仍然是必不可少的，因为相关证据可能无法以预先评价的格式获得。
