Norwegian Institute of Public Health, Oslo, Norway.
School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK.
BMC Med Res Methodol. 2019 Jun 4;19(1):113. doi: 10.1186/s12874-019-0728-6.
Qualitative evidence synthesis is increasingly used alongside reviews of effectiveness to inform guidelines and other decisions. To support this use, the GRADE-CERQual approach was developed to assess and communicate the confidence we have in findings from reviews of qualitative research. One component of this approach requires an appraisal of the methodological limitations of studies contributing data to a review finding. Diverse critical appraisal tools for qualitative research are currently being used. However, it is unclear which tool is most appropriate for informing a GRADE-CERQual assessment of confidence.
We searched for tools that were explicitly intended for critically appraising the methodological quality of qualitative research. We searched the reference lists of existing methodological reviews for critical appraisal tools, and also conducted a systematic search in June 2016 for tools published in health science and social science databases. Two reviewers screened identified titles and abstracts, and then screened the full text of potentially relevant articles. One reviewer extracted data from each article and a second reviewer checked the extraction. We used a best-fit framework synthesis approach to code checklist criteria from each identified tool and to organise these into themes.
We identified 102 critical appraisal tools: 71 tools had previously been included in methodological reviews, and 31 tools were identified from our systematic search. Almost half of the tools were published after 2010. Few authors described how their tool was developed, or why a new tool was needed. After coding all criteria, we developed a framework that included 22 themes. None of the tools included all 22 themes. Some themes were included in up to 95 of the tools.
It is problematic that researchers continue to develop new tools without adequately examining the many tools that already exist. Furthermore, the plethora of tools, old and new, indicates a lack of consensus regarding the best tool to use, and an absence of empirical evidence about the most important criteria for assessing the methodological limitations of qualitative research, including in the context of use with GRADE-CERQual.
定性证据综合越来越多地与效果评估综述一起用于为指南和其他决策提供信息。为了支持这种使用，开发了 GRADE-CERQual 方法来评估和交流我们对综述中定性研究结果的信心。该方法的一个组成部分要求评估为综述结果提供数据的研究的方法学局限性。目前正在使用多种用于定性研究的批判性评估工具。然而，尚不清楚哪种工具最适合为 GRADE-CERQual 评估信心提供信息。
我们搜索了专门用于批判性评估定性研究方法学质量的工具。我们在现有的方法学综述的参考文献中搜索了批判性评估工具，并于 2016 年 6 月在健康科学和社会科学数据库中进行了系统搜索。两名评审员筛选了标题和摘要，然后筛选了可能相关文章的全文。一名评审员从每篇文章中提取数据，另一名评审员检查提取内容。我们使用最佳拟合框架综合方法对每个确定工具的清单标准进行编码，并将这些标准组织成主题。
我们确定了 102 种批判性评估工具：71 种工具以前已包含在方法学综述中，31 种工具是从我们的系统搜索中发现的。几乎一半的工具是在 2010 年后出版的。很少有作者描述他们的工具是如何开发的，或者为什么需要新的工具。在对所有标准进行编码后，我们制定了一个框架，其中包括 22 个主题。没有一个工具包含所有 22 个主题。一些主题被多达 95 个工具所包含。
研究人员继续开发新工具而没有充分检查已经存在的许多工具是有问题的。此外，新旧工具的大量存在表明，对于使用 GRADE-CERQual 评估定性研究方法学局限性的最佳工具，以及对于评估定性研究方法学局限性的最重要标准的实证证据缺乏共识。
