Institute of Health Sciences, Department of Physiotherapy, Pain and Exercise Research Luebeck (P.E.R.L), Universität zu Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Germany.
Faculty of Applied Public Health, European University of Applied Sciences, Werftstr. 5, 18057, Rostock, Germany.
BMC Med Res Methodol. 2022 Apr 6;22(1):100. doi: 10.1186/s12874-022-01561-5.
Internal and external validity are the most relevant components when critically appraising randomized controlled trials (RCTs) for systematic reviews. However, there is no gold standard to assess external validity. This might be related to the heterogeneity of the terminology as well as to unclear evidence of the measurement properties of available tools. The aim of this review was to identify tools to assess the external validity of RCTs. It was further, to evaluate the quality of identified tools and to recommend the use of individual tools to assess the external validity of RCTs in future systematic reviews.
A two-phase systematic literature search was performed in four databases: PubMed, Scopus, PsycINFO via OVID, and CINAHL via EBSCO. First, tools to assess the external validity of RCTs were identified. Second, studies investigating the measurement properties of these tools were selected. The measurement properties of each included tool were appraised using an adapted version of the COnsensus based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.
38 publications reporting on the development or validation of 28 included tools were included. For 61% (17/28) of the included tools, there was no evidence for measurement properties. For the remaining tools, reliability was the most frequently assessed property. Reliability was judged as "sufficient" for three tools (very low certainty of evidence). Content validity was rated as "sufficient" for one tool (moderate certainty of evidence).
Based on these results, no available tool can be fully recommended to assess the external validity of RCTs in systematic reviews. Several steps are required to overcome the identified difficulties to either adapt and validate available tools or to develop a better suitable tool.
Prospective registration at Open Science Framework (OSF): https://doi.org/10.17605/OSF.IO/PTG4D .
在进行系统评价时，内部和外部有效性是评价随机对照试验（RCT）最相关的组成部分。然而，目前还没有评估外部有效性的金标准。这可能与术语的异质性以及现有工具的测量特性的证据不明确有关。本研究的目的是确定评估 RCT 外部有效性的工具。进一步评估了已确定工具的质量，并建议在未来的系统评价中使用个别工具来评估 RCT 的外部有效性。
在四个数据库中进行了两阶段的系统文献搜索：PubMed、Scopus、OVID 中的 PsycINFO 和 EBSCO 中的 CINAHL。首先，确定了评估 RCT 外部有效性的工具。其次，选择了研究这些工具测量特性的研究。使用经过改编的健康测量仪器选择共识标准（COSMIN）指南评估每个纳入工具的测量特性。
共纳入 38 篇报告 28 种纳入工具的开发或验证的研究。对于 28 种纳入工具中的 61%（17/28），没有测量特性的证据。对于其余工具，可靠性是评估最频繁的属性。有三个工具的可靠性被判断为“充分”（极低的证据确定性）。有一个工具的内容效度被评为“充分”（中度证据确定性）。
基于这些结果，没有一种现有的工具可以完全推荐用于评估系统评价中 RCT 的外部有效性。需要采取几个步骤来克服已确定的困难，要么是改编和验证现有的工具，要么是开发更合适的工具。
在开放科学框架（OSF）上进行前瞻性注册：https://doi.org/10.17605/OSF.IO/PTG4D。
