Department of Family Medicine, McGill University, 517 Pine Avenue West, Montreal, QC, Canada H2W 1S4.
Int J Nurs Stud. 2012 Jan;49(1):47-53. doi: 10.1016/j.ijnurstu.2011.07.002. Epub 2011 Aug 10.
Systematic literature reviews identify, select, appraise, and synthesize relevant literature on a particular topic. Typically, these reviews examine primary studies based on similar methods, e.g., experimental trials. In contrast, interest in a new form of review, known as mixed studies review (MSR), which includes qualitative, quantitative, and mixed methods studies, is growing. In MSRs, reviewers appraise studies that use different methods allowing them to obtain in-depth answers to complex research questions. However, appraising the quality of studies with different methods remains challenging. To facilitate systematic MSRs, a pilot Mixed Methods Appraisal Tool (MMAT) has been developed at McGill University (a checklist and a tutorial), which can be used to concurrently appraise the methodological quality of qualitative, quantitative, and mixed methods studies.
The purpose of the present study is to test the reliability and efficiency of a pilot version of the MMAT.
The Center for Participatory Research at McGill conducted a systematic MSR on the benefits of Participatory Research (PR). Thirty-two PR evaluation studies were appraised by two independent reviewers using the pilot MMAT. Among these, 11 (34%) involved nurses as researchers or research partners. Appraisal time was measured to assess efficiency. Inter-rater reliability was assessed by calculating a kappa statistic based on dichotomized responses for each criterion. An appraisal score was determined for each study, which allowed the calculation of an overall intra-class correlation.
On average, it took 14 min to appraise a study (excluding the initial reading of articles). Agreement between reviewers was moderate to perfect with regards to MMAT criteria, and substantial with respect to the overall quality score of appraised studies.
The MMAT is unique, thus the reliability of the pilot MMAT is promising, and encourages further development.
系统文献综述旨在识别、选择、评价和综合特定主题的相关文献。通常，这些综述基于类似的方法（例如，实验性试验）来检查主要研究。相比之下，对一种新的综述形式，即混合研究综述（MSR）的兴趣正在增加。在 MSR 中，评审员评价使用不同方法的研究，从而能够对复杂的研究问题获得深入的答案。然而，评价具有不同方法的研究的质量仍然具有挑战性。为了促进系统的 MSR，麦吉尔大学开发了一个混合方法评估工具（MMAT）的试点版本（检查表和教程），该工具可用于同时评估定性、定量和混合方法研究的方法质量。
本研究的目的是测试 MMAT 试点版本的可靠性和效率。
麦吉尔大学参与式研究中心对参与式研究（PR）的益处进行了系统的 MSR。使用试点 MMAT，两名独立评审员对 32 项 PR 评估研究进行了评价。其中，11 项（34%）研究涉及护士作为研究人员或研究伙伴。评估时间用于评估效率。通过基于每个标准的二分响应计算kappa 统计量来评估组内可靠性。为每项研究确定了评估分数，这允许计算总体组内相关系数。
平均而言，评估一项研究需要 14 分钟（不包括阅读文章的初始时间）。对于 MMAT 标准，评审员之间的一致性为中等至完美，对于评估研究的总体质量评分，一致性为强。
MMAT 是独特的，因此试点 MMAT 的可靠性很有希望，并鼓励进一步开发。
