Huang Yidi, Yuan William, Kohane Isaac S, Beaulieu-Jones Brett K
Department of Biomedical Informatics, Harvard Medical School, Countway Library, Boston, Massachusetts, USA.
JAMIA Open. 2021 Jun 16;4(2):ooab045. doi: 10.1093/jamiaopen/ooab045. eCollection 2021 Apr.
Case-control study designs are commonly used in retrospective analyses of real-world evidence (RWE). Due to the increasingly wide availability of RWE, it can be difficult to determine whether findings are robust or the result of testing multiple hypotheses.
We investigate the potential effects of modifying cohort definitions in a case-control association study between depression and type 2 diabetes mellitus. We used a large (>75 million individuals) de-identified administrative claims database to observe the effects of minor changes to the requirements of glucose and hemoglobin A1c tests in the control group.
We found that small permutations to the criteria used to define the control population result in significant shifts in both the demographic structure of the identified cohort as well as the odds ratio of association. These differences remain present when testing against age- and sex-matched controls.
Analyses of RWE need to be carefully designed to avoid issues of multiple testing. Minor changes to control cohorts can lead to significantly different results and have the potential to alter even prospective studies through selection bias.
We believe this work offers strong support for the need for robust guidelines, best practices, and regulations around the use of observational RWE for clinical or regulatory decision-making.
病例对照研究设计常用于真实世界证据（RWE）的回顾性分析。由于RWE的可得性日益广泛，很难确定研究结果是可靠的，还是多次检验假设的结果。
我们在抑郁症与2型糖尿病的病例对照关联研究中，调查修改队列定义的潜在影响。我们使用了一个大型（超过7500万人）匿名管理索赔数据库，以观察对照组中血糖和糖化血红蛋白检测要求的微小变化所产生的影响。
我们发现，用于定义对照人群的标准的微小排列，会导致所确定队列的人口结构以及关联比值比发生显著变化。在与年龄和性别匹配的对照组进行检验时，这些差异依然存在。
对RWE的分析需要精心设计，以避免多重检验问题。对照队列的微小变化可能导致显著不同的结果，甚至有可能通过选择偏倚改变前瞻性研究。
我们认为，这项工作有力支持了围绕使用观察性RWE进行临床或监管决策制定强有力的指南、最佳实践和法规的必要性。
