Wakabayashi Ryozo, Hirano Takahiro, Koga Tadashi, Kuwatsuru Ryohei
Clinical Study Support, Inc., Nagoya, Japan.
Real-World Evidence and Data Assessment (READS), Graduate School of Medicine, Juntendo University, 2-1-1 Hongo, Bunkyo-ku, Tokyo, 113-8421, Japan.
Drugs Real World Outcomes. 2023 Jun;10(2):235-247. doi: 10.1007/s40801-023-00353-7. Epub 2023 Feb 1.
Use of real-world evidence (RWE) has been limited for evaluating effectiveness because of the lack of confidence in its reliability. Examining whether a rigorously designed observational study using real-world data (RWD) can reproduce the results of a randomized controlled trial (RCT) will provide insights into the implementation of high-quality RWE studies that can produce valid conclusions.
We aimed to replicate published RCTs using a Japanese claims and health checkup database and examine whether the emulated RWE studies' results agree with those of the original RCTs.
We selected three RCTs on diabetes medications for replication in patients with type 2 diabetes. The study outcome was either the change or percentage change in HbA1c levels from baseline. We designed three observational studies using the RWD to mimic the critical study elements of the respective RCTs as closely as possible. We performed 1:1 propensity score nearest-neighbor matching to balance the groups for potential confounders. The differences in outcomes between the groups and their 95% confidence intervals (CIs) were calculated in each RWE study, and the results were compared with those of the RCT.
Patient characteristics, such as age, sex, and duration of diabetes, differed between the RWE studies and RCTs. In Trial 1 emulation, the percentage changes in HbA1c levels were larger in the treatment group than in the comparator group (difference -6.21, 95% confidence interval (CI) -11.01 to -1.40). In Trial 2, the change in HbA1c level was larger in the treatment group (difference -0.01; 95% CI -0.25 to 0.23), and in Trial 3, it was smaller in the treatment group (difference 0.46; 95% CI -0.01 to 0.94). These results did not show regulatory or estimate agreement with the RCTs.
None of the three emulated RWE studies using this claims and health checkup database reproduced the same conclusions as the RCTs. These discrepancies could largely be attributed to design differences between RWE studies and RCTs, primarily due to the lack of necessary data in the database. This particular RWD source may not be the best fit for evaluating treatment effects using laboratory data as the study outcome.
由于对真实世界证据（RWE）的可靠性缺乏信心，其在评估有效性方面的应用一直受到限制。检验一项使用真实世界数据（RWD）进行严格设计的观察性研究是否能够重现随机对照试验（RCT）的结果，将为开展能够得出有效结论的高质量RWE研究提供见解。
我们旨在使用日本的医保理赔和健康检查数据库复制已发表的随机对照试验，并检验模拟的RWE研究结果是否与原始随机对照试验的结果一致。
我们选择了三项关于糖尿病药物治疗2型糖尿病患者的随机对照试验进行复制。研究结局为糖化血红蛋白（HbA1c）水平相对于基线的变化或变化百分比。我们使用真实世界数据设计了三项观察性研究，尽可能紧密地模拟各自随机对照试验的关键研究要素。我们进行了1:1倾向评分最近邻匹配，以平衡各潜在混杂因素组。在每项RWE研究中计算了组间结局差异及其95%置信区间（CI），并将结果与随机对照试验的结果进行比较。
RWE研究与随机对照试验在患者特征方面存在差异，如年龄、性别和糖尿病病程。在试验1模拟中，治疗组HbA1c水平的变化百分比高于对照（差异为-6.21，95%置信区间（CI）为-11.01至-1.40）。在试验2中，治疗组HbA1c水平变化更大（差异为-0.01；95%CI为-0.25至0.23），而在试验3中，治疗组变化更小（差异为0.46；95%CI为-0.01至0.94）。这些结果在监管或估计方面均与随机对照试验不一致。
使用该医保理赔和健康检查数据库进行的三项模拟RWE研究均未得出与随机对照试验相同的结论。这些差异很大程度上可归因于RWE研究与随机对照试验之间的设计差异，主要原因是数据库中缺乏必要数据。这一特定的真实世界数据源可能并非最适合以实验室数据作为研究结局来评估治疗效果。
