Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada.
School of Nursing, Queen's University, Kingston, Ontario, Canada.
Syst Rev. 2019 Nov 7;8(1):266. doi: 10.1186/s13643-019-1161-y.
Experimental designs for evaluating knowledge translation (KT) interventions can provide strong estimates of effectiveness but offer limited insight into how the intervention worked. Consequently, process evaluations have been used to explore the causal mechanisms at work; however, there are limited standards to guide this work. This study synthesizes current evidence of KT process evaluations to provide future methodological recommendations.
Peer-reviewed search strategies were developed by a health research librarian. Studies had to be in English, published since 1996, and were not excluded based on design. Studies had to (1) be a process evaluation of a KT intervention study in primary health, (2) be a primary research study, and (3) include a licensed healthcare professional delivering or receiving the intervention. A two-step, two-person hybrid screening approach was used for study inclusion with inter-rater reliability ranging from 94 to 95%. Data on study design, data collection, theoretical influences, and approaches used to evaluate the KT intervention, analysis, and outcomes were extracted by two reviewers. Methodological quality was assessed with the Mixed Methods Appraisal Tool (MMAT).
Of the 20,968 articles screened, 226 studies fit our inclusion criteria. The majority of process evaluations used qualitative forms of data collection (43.4%) and individual interviews as the predominant data collection method. 72.1% of studies evaluated barriers and/or facilitators to implementation. 59.7% of process evaluations were stand-alone evaluations. The timing of data collection varied widely with post-intervention data collection being the most frequent (46.0%). Only 38.1% of the studies were informed by theory. Furthermore, 38.9% of studies had MMAT scores of 50 or less indicating poor methodological quality.
There is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced through complementary process evaluations. However, this systematic review found that process evaluations are of mixed quality and lack theoretical guidance. Most process evaluation data collection occurred post-intervention undermining the ability to evaluate the process of implementation. Strong science and methodological guidance is needed to underpin and guide the design and execution of process evaluations in KT science.
This study is not registered with PROSPERO.
评估知识转化 (KT) 干预措施的实验设计可以提供有效性的有力估计，但对干预措施的工作方式提供的了解有限。因此，已经使用过程评估来探索作用中的因果机制；然而，指导这项工作的标准有限。本研究综合了当前 KT 过程评估的证据，为未来提供方法学建议。
由健康研究图书管理员制定了同行评审的搜索策略。研究必须为英文，发表于 1996 年以后，且不基于设计进行排除。研究必须（1）为初级卫生保健中 KT 干预研究的过程评估，（2）为主要研究，并且（3）包括提供或接受干预措施的持照医疗保健专业人员。使用两步、双人混合筛选方法进行研究纳入，评分者间的可靠性范围为 94 到 95%。两名审查员提取关于研究设计、数据收集、理论影响以及评估 KT 干预措施、分析和结果的方法的数据。使用混合方法评估工具 (MMAT) 评估方法学质量。
在筛选出的 20968 篇文章中，有 226 篇符合纳入标准。大多数过程评估使用定性形式的数据收集（43.4%），并以个体访谈作为主要的数据收集方法。72.1%的研究评估了实施的障碍和/或促进因素。59.7%的过程评估是独立评估。数据收集的时间差异很大，以干预后数据收集最为常见（46.0%）。只有 38.1%的研究有理论依据。此外，38.9%的研究的 MMAT 得分为 50 或更低，表明方法学质量较差。
人们普遍认为，通过补充性的过程评估，将大大提高 KT 干预措施的定量试验的可推广性。然而，本系统评价发现，过程评估的质量参差不齐，缺乏理论指导。大多数过程评估的数据收集发生在干预后，这削弱了评估实施过程的能力。需要强有力的科学和方法学指导，为 KT 科学中的过程评估的设计和执行提供依据和指导。
