Janssen Research & Development, Titusville, USA.
Flatiron Health, Inc, 233 Spring Street, New York, NY, 10013, USA.
BMC Med Res Methodol. 2023 Aug 24;23(1):193. doi: 10.1186/s12874-023-02002-7.
Novel precision medicine therapeutics target increasingly granular, genomically-defined populations. Rare sub-groups make it challenging to study within a clinical trial or single real-world data (RWD) source; therefore, pooling from disparate sources of RWD may be required for feasibility. Heterogeneity assessment for pooled data is particularly complex when contrasting a pooled real-world comparator cohort (rwCC) with a single-arm clinical trial (SAT), because the individual comparisons are not independent as all compare a rwCC to the same SAT. Our objective was to develop a methodological framework for pooling RWD focused on the rwCC use case, and simulate novel approaches of heterogeneity assessment, especially for small datasets.
We present a framework with the following steps: pre-specification, assessment of dataset eligibility, and outcome analyses (including assessment of outcome heterogeneity). We then simulated heterogeneity assessments for a binary response outcome in a SAT compared to two rwCCs, using standard methods for meta-analysis, and an Adjusted Cochran's Q test, and directly comparing the individual participant data (IPD) from the rwCCs.
We found identical power to detect a true difference for the adjusted Cochran's Q test and the IPD method, with both approaches superior to a standard Cochran's Q test. When assessing the impact of heterogeneity in the null scenario of no difference between the SAT and rwCCs, a lack of statistical power led to Type 1 error inflation. Similarly, in the alternative scenario of a true difference between SAT and rwCCs, we found substantial Type 2 error, with underpowered heterogeneity testing leading to underestimation of the treatment effect.
We developed a methodological framework for pooling RWD sources in the context of designing a rwCC for a SAT. When testing for heterogeneity during this process, the adjusted Cochran's Q test matches the statistical power of IPD heterogeneity testing. Limitations of quantitative heterogeneity testing in protecting against Type 1 or Type 2 error indicate these tests are best used descriptively, and after careful selection of datasets based on clinical/data considerations. We hope these findings will facilitate the rigorous pooling of RWD to unlock insights to benefit oncology patients.
新型精准医学治疗方法针对的是越来越细粒度的、基于基因组定义的人群。在临床试验或单一真实世界数据 (RWD) 来源中，罕见亚组的研究极具挑战性；因此，可能需要从不同的 RWD 来源中进行汇总以确保可行性。当将汇总的真实世界对照队列 (rwCC) 与单臂临床试验 (SAT) 进行对比时，汇总数据的异质性评估特别复杂，因为个体比较并不独立，因为所有比较都是将 rwCC 与同一个 SAT 进行比较。我们的目标是开发一种专注于 rwCC 用例的汇总 RWD 的方法框架，并模拟新的异质性评估方法，特别是对于小数据集。
我们提出了一个框架，包括以下步骤：预先指定、数据集资格评估和结果分析（包括结果异质性评估）。然后，我们使用荟萃分析的标准方法和调整后的 Cochran's Q 检验，以及直接比较 rwCC 的个体参与者数据 (IPD)，对 SAT 与两个 rwCC 相比的二分类反应结局进行了异质性评估模拟。
我们发现，调整后的 Cochran's Q 检验和 IPD 方法检测真实差异的功效相同，这两种方法都优于标准 Cochran's Q 检验。当评估在 SAT 和 rwCC 之间没有差异的零假设情况下异质性的影响时，缺乏统计学功效导致了Ⅰ类错误膨胀。同样，在 SAT 和 rwCC 之间存在真实差异的替代假设下，我们发现存在大量的Ⅱ类错误，由于异质性检验功效不足，导致对治疗效果的低估。
我们开发了一种在设计 SAT 用 rwCC 的背景下汇总 RWD 来源的方法框架。在这个过程中进行异质性测试时，调整后的 Cochran's Q 检验与 IPD 异质性测试的统计功效相匹配。定量异质性测试在防范Ⅰ类或Ⅱ类错误方面的局限性表明，这些测试最好是在基于临床/数据考虑对数据集进行仔细选择后，以描述性方式使用。我们希望这些发现将促进对 RWD 的严格汇总，以挖掘对肿瘤学患者有益的见解。
