Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts, USA.
HEOR Department, Cytel Inc, Toronto, Quebec, Canada.
BMJ Open. 2021 Mar 24;11(3):e043961. doi: 10.1136/bmjopen-2020-043961.
To determine whether assessment tools for non-randomised studies (NRS) address critical elements that influence the validity of NRS findings for comparative safety and effectiveness of medications.
Systematic review and Delphi survey.
We searched PubMed, Embase, Google, bibliographies of reviews and websites of influential organisations from inception to November 2019. In parallel, we conducted a Delphi survey among the International Society for Pharmacoepidemiology Comparative Effectiveness Research Special Interest Group to identify key methodological challenges for NRS of medications. We created a framework consisting of the reported methodological challenges to evaluate the selected NRS tools.
Checklists or scales assessing NRS.
Two reviewers extracted general information and content data related to the prespecified framework.
Of 44 tools reviewed, 48% (n=21) assess multiple NRS designs, while other tools specifically addressed case-control (n=12, 27%) or cohort studies (n=11, 25%) only. Response rate to the Delphi survey was 73% (35 out of 48 content experts), and a consensus was reached in only two rounds. Most tools evaluated methods for selecting study participants (n=43, 98%), although only one addressed selection bias due to depletion of susceptibles (2%). Many tools addressed the measurement of exposure and outcome (n=40, 91%), and measurement and control for confounders (n=40, 91%). Most tools have at least one item/question on design-specific sources of bias (n=40, 91%), but only a few investigate reverse causation (n=8, 18%), detection bias (n=4, 9%), time-related bias (n=3, 7%), lack of new-user design (n=2, 5%) or active comparator design (n=0). Few tools address the appropriateness of statistical analyses (n=15, 34%), methods for assessing internal (n=15, 34%) or external validity (n=11, 25%) and statistical uncertainty in the findings (n=21, 48%). None of the reviewed tools investigated all the methodological domains and subdomains.
The acknowledgement of major design-specific sources of bias (eg, lack of new-user design, lack of active comparator design, time-related bias, depletion of susceptibles, reverse causation) and statistical assessment of internal and external validity is currently not sufficiently addressed in most of the existing tools. These critical elements should be integrated to systematically investigate the validity of NRS on comparative safety and effectiveness of medications. SYSTEMATIC REVIEW PROTOCOL AND REGISTRATION: https://osf.io/es65q.
确定非随机研究（NRS）评估工具是否涉及影响药物比较安全性和有效性的 NRS 结果有效性的关键因素。
系统评价和 Delphi 调查。
我们从成立到 2019 年 11 月在 PubMed、Embase、Google、综述的参考文献和有影响力的组织网站上进行了搜索。同时，我们在国际药物流行病学比较有效性研究学会利益相关者小组中进行了 Delphi 调查，以确定药物 NRS 的关键方法学挑战。我们创建了一个框架，其中包含报告的方法学挑战，以评估所选的 NRS 工具。
评估 NRS 的清单或量表。
两位审查员提取了与预定义框架相关的一般信息和内容数据。
在审查的 44 种工具中，48%（n=21）评估了多种 NRS 设计，而其他工具则专门针对病例对照（n=12，27%）或队列研究（n=11，25%）。德尔菲调查的回复率为 73%（35 位内容专家中的 35 位），仅在两轮中达成共识。大多数工具评估了研究参与者选择方法（n=43，98%），尽管只有一种工具解决了由于易感人群枯竭导致的选择偏倚（2%）。许多工具评估了暴露和结局的测量（n=40，91%），以及混杂因素的测量和控制（n=40，91%）。大多数工具都有至少一个针对特定设计偏倚源的项目/问题（n=40，91%），但只有少数工具调查了反向因果关系（n=8，18%）、检测偏倚（n=4，9%）、时间相关偏倚（n=3，7%）、缺乏新用户设计（n=2，5%）或活性对照设计（n=0）。很少有工具涉及统计分析的适当性（n=15，34%）、内部（n=15，34%）或外部有效性（n=11，25%）以及研究结果的统计不确定性（n=21，48%）。审查的工具中没有一个调查了所有的方法学领域和子领域。
目前，大多数现有工具都没有充分考虑到主要的特定设计偏倚源（例如，缺乏新用户设计、缺乏活性对照设计、时间相关偏倚、易感人群枯竭、反向因果关系）和内部及外部有效性的统计评估。这些关键要素应纳入其中，以系统地研究药物比较安全性和有效性的 NRS 结果的有效性。
